how do I get modules to stop eating memory

Skip Montanaro skip at pobox.com
Mon Jul 2 11:18:33 EDT 2001


    Charles> - Prior to importing BaseHTTPServer the memory usage was 1432. 
    Charles> Shouldn't Python's memory usage have gone down to that number,
    Charles> or at least down some?

That depends entirely upon two things: 

    1. How the malloc package you're using behaves in the face of newly
       freed memory.

    2. Whether or not it thinks the memory can be freed.

Very basically, when you ask to malloc a chunk of memory and malloc can't
locate a sufficiently large free block of memory in the chunks already
allocated, it calls the brk system call (or the sbrk library call, which
calls sbrk) to get more.  (Some now use mmap, at least for very large
blocks.  That just complicates things.) The brk call just extends the
process's heap to some new address.  If I then malloc some memory from that
block and free *all of it*, the malloc package can *if it so desires* free
that block.  If any memory is still allocated in that block, it can't be
given back to the system.  In particular, the heap is treated as a stack in
most (all?)  systems, so only the topmost block of memory brk'd from the
system is eligible to be given back (item #2 above).  Then, and only then,
will you see your process's memory usage (as displayed by top or ps) get
smaller.  Many malloc packages never release memory back to the system (item
#1 above).

For more details, read the GNU malloc source code.  Now, there's some
fun... ;-)

-- 
Skip Montanaro (skip at pobox.com)
(847)971-7098




More information about the Python-list mailing list