[Python-Dev] update on memory leaks in 2.2

[Guido van Rossum]

> Guido van Rossum wrote:
> 
> > What's a potential leak?  Could it be that these are in cycles?  The
> > reported blocks are all weak reference objects; most of then are
> > allocated by add_subclass(), which uses weak refs for the list of
> > subclasses of a given (new-style) class.  I suppose most weak refs are
> > in cycles (else there would be no need for weak refs).

Neal replied (in private email):

> From page 157 of this doc:
> 
> http://www.rational.com/support/documentation/manuals/unixsuites/pdf/purify/purify.pdf
> 
> A PLK message describes heap memory that you might have
> leaked. You have pointers only to the middle of the region. PLK is a
> warning message.
> 
> Memory in use can sometimes appear as a PLK if the pointer
> returned by malloc is offset. A common cause is referencing a
> substring within a large string. Another example is when a
> pointer to a C++ object is cast to the second or later base class of a
> multiply-inherited object. It is offset past the other base class
> objects.
> 
> Truly leaked memory can sometimes appear as a PLK, if some
> non-pointer integer within the program space, when interpreted
> as a pointer, points within an otherwise leaked block of memory.
> This is rather rare. Inspect the code to differentiate between these
> causes of PLK reports.

Thanks, that's very helpful!  Here's how I now understand what's
happened.  Weak references are garbage collected, which means that the
malloc pointer points 12 bytes *before* the start of the object as the
rest of Python knows about it.  Objects in the free list are linked
together via their object pointer, so they are only known via a
pointer into their "middle".  We don't see this warning for other
GC'ed objects, because they are in one of the collector's linked
lists, which point to the start of the GC header which is the
malloc'ed pointer.  But freed weak references are not known to the
collector!

So I'm confident that these are not leaks.

--Guido van Rossum (home page: http://www.python.org/~guido/)