Tracking down memory leaks?

[Robert]

MKoool wrote:

> I have an application with one function called "compute", which given a
> filename, goes through that file and performs various statistical
> analyses.  It uses arrays extensively and loops alot.  it prints the
> results of it's statistical significance tests to standard out.  Since
> the compute function returns and I think no variables of global scope
> are being used, I would think that when it does, all memory returns
> back to the operating system.
> 
> Instead, what I see is that every iteration uses several megs more.
> For example, python uses 52 megs when starting out, it goes through
> several iterations and I'm suddenly using more than 500 megs of ram.
> 
> Does anyone have any pointers on how to figure out what I'm doing
> wrong?
> 

if gc.collect() doesn't help:

maybe objects of extension libs are not freed correctly.

And Python has a real skeleton in the cupboard:

a known problem with python object/libs when classes with __del__ are 
involved ( Once suffered myself from such tremendous "unexplainable" 
memory blow up until I found this "del gc.garbage[:]" remedy:

<http://www.python.org/doc/current/lib/module-gc.html>
garbage
     A list of objects which the collector found to be unreachable but 
could not be freed (uncollectable objects). By default, this list 
contains only objects with __del__() methods.3.1Objects that have 
__del__() methods and are part of a reference cycle cause the entire 
reference cycle to be uncollectable, including objects not necessarily 
in the cycle but reachable only from it. Python doesn't collect such 
cycles automatically because, in general, it isn't possible for Python 
to guess a safe order in which to run the __del__() methods. If you know 
a safe order, you can force the issue by examining the garbage list, and 
explicitly breaking cycles due to your objects within the list. Note 
that these objects are kept alive even so by virtue of being in the 
garbage list, so they should be removed from garbage too. For example, 
after breaking cycles, do del gc.garbage[:] to empty the list. It's 
generally better to avoid the issue by not creating cycles containing 
objects with __del__() methods, and garbage can be examined in that case 
to verify that no such cycles are being created.


Robert