when does python free memory?

[Delaney, Timothy]

> From: reanes at banta-im.com [mailto:reanes at banta-im.com]

> The first number that comes out of the "print memsize" statement is
> the pid, the second is the amount of memory used.  It doesn't seem to
> be actually freeing the memory that was allocated to variable z.  Now
> I played around with it a little more and if I reallocate another list
> it does re-use that memory:
> 
> So apparently python knows that the memory formerly allocated to z is
> available, but it doesn't tell the OS.  Is this the case, and if so,
> is there any way around it?  I have an application that needs to run
> 80-150 python processes that can stay around for 15min to an hour, so
> I'd like them not to hold onto the maximum amount of memory that they
> use throughout the whole life of the process.

This is normal with most applications - they grow, but rarely shrink their
total used memory. It depends primarily on the C library memory allocator
(malloc) implementation for the platform.

If you want 80-150 python processes running, each with their own copy of the
python VM, you are going to end up with 80-150 copies of the VM using up
memory. This in itself may or may not be significant, however it leads to
the same problem - each instance of the VM will grow its memory usage, but
usually not shrink.

Normally a better way would be to either have each "process" running as a
different Python thread, or build Python as a shared library - then the
"processes" would then be able to reuse memory freed by a different
"process" (at least, I think this would work with a shared library - it will
definitely work with threads) leading to a smaller amount of memory being
used overall.

Tim Delaney