Leakage vs Cyclops

[Robin Becker]

In article <m3ptrd3ftw.fsf at mira.informatik.hu-berlin.de>, Martin v.
Löwis <martin at v.loewis.de> writes
>Robin Becker <robin at jessikat.fsnet.co.uk> writes:
>
>> this leak is present with 1.52 so it's not a function of 2.2/gc. Are you
>> saying there are cycles/leaks that are not caught by inspecting toplevel
>> module variables?
>
>"There are" is a big general here. It is certainly possible to have
>such leaks; a simple error in INCREF/DECREF could easily trigger such
>problems. Of course, the Python core never should have such leaks, and
>likely has none that have survived since 1.5.2.
>
>However, there are other causes of memory consumption, too, for
>example lists that grow longer and longer without ever being part of a
>cycle.
>
>The most notable such thing is the dictionary of interned strings,
>which isn't even reachable through a module level variable. This is by
>design; this design will change only in Python 2.3 (where interned
>strings become mortal).
>
>If you meant "toplevel module" literally, then there are certainly way
>more things unreachable: unless sys is imported, you won't find
>sys.modules from the toplevel module. 
>
>Even if you add sys.modules to your object roots, there are still
>unreachable non-garbage objects: C modules may hold them in global
>variables, and not expose them. There are many such cases in Python
>1.5.2, and still a lot in newer versions (although more and more
>objects get exposed in more recent releases).
>
>Regards,
>Martin
>
I can certainly see how such objects could be created as a loop and then
cast off from their root, but I thought that gc would catch these. As
you say it's possible some unsuspected list is growing ever longer.

In fact using python -v I can see that there are (at least) 10 floats
created per page which aren't being removed which suggests something
isn't right. Our C extensions aren't to blame as I can remove them and
still have the same problem.

Using print gc.garbage indicates that I have no finalizer loops so I
guess I need to do the kind of detailed analysis you suggest.
-- 
Robin Becker