[Python-Dev] CALL_ATTR patch (was: 2.3b1 release)

[Tim Peters]

[Michael Hudson]
> ...
> And then I find out running test_descr in a loop leaks 166(!)
> references *anyway* (run the attached blah.py in a debug build, and
> some print test or other that I don't understand fails when run like
> this, so I fiddled it)
>
> Should I have been expecting this, or is it time for some ref-leak
> hunting?  Actually, the process seems to grow -- slowly -- when you
> loop, so it's probably a real problem.  I gather it's best to ask
> about this sort of thing when Tim is off sick from work :-)

Only if you want help <wink>.  Since Zope has worse leak problems than
Python, Zope testing has more tools to dig into it:

    http://cvs.zope.org/Zope3/test.py

That's the Zope3 unittest test driver.  The TrackRefs class is a handy
little beast, which uses a debug build's sys.getobjects() to keep track of
how many objects of each *type* exist across test runs, and how many
refcounts total they account for.  After the first few iterations, TrackRefs
doesn't itself distort the results it prints.  It will tell you the types of
the objects that are leaking, and also whether objects are actually leaking,
or that merely refcounts to existing objects are leaking (e.g., a common
kind of "leak" is to forget to decref Py_None, but memory doesn't actually
grow then).

Do use 2.3 if you try this.  The "list of all objects" before 2.3 turned out
to be missing many objects that tend to leak (None, booleans, any number of
type objects, essentially all statically allocated objects), and far fewer
are missing in 2.3.

It's typically easy to find out which types of objects are leaking this way.
Then it's typical to add more code to TraceRefs.update() to get more
details.  In Zope use, it's surprising how often that still left us
scratching our heads!  But Zope has lots of extension types coded in C,
massive interconnections among them, and none of them originally
participated in cyclic gc.  Python leaks are usually easier to plug (for
example, in Python, they're almost always in new code).