memory recycling/garbage collecting problem

[Chris Rebert]

On Mon, Feb 16, 2009 at 9:21 PM, Yuanxin Xi <xi11west at gmail.com> wrote:
> I'm having some problems with the memory recycling/garbage collecting
> of the following testing code:
>
>>>> a=[str(i) for i in xrange(10000000)]
> This takes 635m/552m/2044 memory (VIRT/RES/SHR)
>
>>>> b={}
>>>> for i in xrange(10000000):
> ...     b[str(i)]=i
>
> Then the memory usage increased to 1726m/1.6g/2048
>
>>>> del b
> I expect the memory usage drop to the ammount before b was
> created(635m/552m/2044), but it's actually 1341m/1.2g/2048
>
> Could anyone please explain why this happens?  It seems some memory
> are not freed.  I'm running into problems with this as my program is
> very memory cosuming and need to frequently free some object to reuse
> the memory.  What is the best way to free the memory of b completely
> (i.e. back to the status as b was created)?  I'm using Python 2.5.2

My understanding is that for efficiency purposes Python hangs on to
the extra memory even after the object has been GC-ed and doesn't give
it back to the OS right away. This shouldn't matter to your program
though, as the "extra" memory will still be available for its use. And
assuming you're using CPython and a/b isn't referred to by anything
else, I believe they should be GC-ed immediately by the refcount
system after the 'del' s executed. So, in theory you shouldn't need to
worry about any of this, unless your example is not accurate and
you're dealing with cyclical references or C objects or something.

Cheers,
Chris

-- 
Follow the path of the Iguana...
http://rebertia.com