Can I get the total memory usage of Python?

Tim Peters tim.one at comcast.net
Fri May 23 13:17:18 EDT 2003


[Cameron Laird]
> Right; it's a general principle of common OSs that
> t's shockingly hairy to find out what is *really*
> going on in memory.
>
> What I meant, although far too elliptically, was just
> at the pymalloc level.

You may have meant that <wink>, but note that pymalloc only intercepts
requests for "small" objects, and only for those small-object subsystems
that explicitly choose to use pymalloc.  For example, memory for a large
list comes straight from the platform malloc, and even though ints are small
objects they use their own layer of cruft around the platform malloc.  IOW,
there is no choke point for memory allocation in Python above the platform
malloc.

> No, that doesn't correspond to real memory use, as seen "from the
> outside"; my own intentions for memory introspection are satisfied by
> anything which reliably enumerates my use of memory, even if it's
> inaccurately quantified.  It's enough for me to have a handle on how
> many and which objects I'm asking the system to juggle.

That's a different question:  malloc and pymalloc requests are untyped --
they have no idea what the memory is used for.  You may like the stats you
can get in a Python COUNT_ALLOCS build instead:  that keeps track of the
current total number of allocations, frees, and highwater mark (max value of
#allocations - #frees over time) on a per-type basis (so, e.g., the stats
for ints are distinct from the stats for floats).  That's been available
"forever", but it's a special build because it's not particularly cheap to
keep track of this stuff.






More information about the Python-list mailing list