Where'd my memory go? (was Re: [Python-Dev] Dictionary sparseness)

[Tim Peters]

[Skip Montanaro]
> Here's a thought.  Debug builds appear to now add a getobjects method to
> sys.

Yes, although that isn't new -- it's been there forever (read
Misc/SpecialBuilds.txt).

> Would it be possible to also add another method to sys (also only
> available on debug builds) which knows just enough about basic builtin
> object types to say a little about how much space it's consuming?

Marc-Andre has something like that in mxTools already (his sizeof()
function).

Note also the COUNT_ALLOCS special build, which saves info about total # of
allocations, deallocations, and highwater mark per type, made available via
sys.getcounts().  The nifty thing about COUNT_ALLOCS is that you can enable
it in a release build (it doesn't rely on the debug-build changes to the
layout of PyObject).

Stuff all these things miss (even pymalloc, because it isn't asked for the
memory) include the immortal and unbounded int freelist, the I&U float FL,
and the immortal but bounded frameobject FL.  Do, e.g., range(2000000) (as
someone did on c.l.py last week), and about 24MB "goes missing" until the
program shuts down (it's sitting in the int FL).  Note that pymalloc never
returns its "arenas" to the system either.