Hunting a memory leak

[Francesc Alted]

On 2003-08-29, Michael Hudson <mwh at python.net> wrote:
> Debian User <falted at inspiron.openlc.org> writes:
>
>> I'm trying to discover a memory leak on a program of mine. I've taken
>> several approaches, but the leak still resists to appear.
>> 
>> First of all, I've tried to use the garbage collector to look for
>> uncollectable objects.
>
> [snip]
>
>> Then I've taken an approach that I've seen in python developers list
>> contributed by Walter Dörwald, that basically consists in creating a
>> debug version of python, create a unitest with the leaking code, and
>> modify the unittest.py to extract the increment of total reference
>> counting in that code (see
>> http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1770868).
>
> Well, somewhere in that same thread are various references to a
> TrackRefs class.  Have you tried using that?  It should tell you what
> type of object is leaking, which is a good start.
>
>> With that, I see that my reference count grows by one each time the
>> test execute. But the problem is: is there some way to look at the
>> object (or make a memory dump) that is leaking?.
>
> See above :-)
>
>> I've used valgrind (http://developer.kde.org/~sewardj/) to see if it
>> could detect the leak. In fact, it detects a bunch of them, but I am
>> afraid that they are not related with the leak I'm looking for. I am
>> saying that because, when I loop over my leaky code, valgrind always
>> report the same amount of leaky memory, independently of the number of
>> iterations (while top is telling me that memory use is growing!).
>
> There are various things (interned strings, f'ex) that always tend to
> be alive at the end of a Python program: these are only leaks in a
> very warped sense.
>
> I don't know if there's a way to get vaglrind to tell you what's
> allocated but not deallocated between two arbitrary points of program
> execution.
>
>> My code uses extension modules in C, so I am afraid this does not
>> contribute to alleviate the problem.
>
> Well, in all likelyhood, the bug is IN the C extension module.  Have
> you tried stepping through the code in a debugger?  Sometime's that's
> a good way of spotting a logic error.
>
>> I am sorry, but I cannot be more explicit about the code because it
>> is quite complex (it is the PyTables package, http://pytables.sf.net),
>> and I was unable to make a simple example to be published
>> here. However, if anyone is tempted to have a look at the code, you
>> can download it from
>> (http://sourceforge.net/project/showfiles.php?group_id=63486). I am
>> attaching a unittest that exposes the leak.
>> 
>> I am a bit desperate. Any hint?
>
> Not really.  Try using TrackRefs.
>
> Cheers,
> mwh
>