memory leak with large list??

[Stanley]

I should start out by saying that I am using the linux command 'ps' to
check memory usage, and I am not entirely sure of the implications of
this.

>>> def memcheck():
>>> 	command_text = 'ps -o pmem,rss,rssize,sz,vsz ' + str(os.getpid())
>>> 	os.system(command_text)


First I check the memory footprint of the process.
%MEM  RSS  RSS    SZ   VSZ
 0.3 1824 1824   734  2936

Then I generate a large list (12 million floats), either using map, a
list comprehension, or preallocating a list, then filling it using a
for loop.


>>> big_list = [float(index) for index in xrange(12000000)]

or 

>>> big_list = map(float, xrange(12000000))

or 

>>> big_list = [0.0] * 12000000
>>> for index in xrange(12000000):
...	    big_list[index] = float(index)


When I check the memory footprint again, it is larger than I would
expect, but I'm not sure what the per element overhead of a list is.

%MEM  RSS  RSS    SZ   VSZ
46.2 237648 237648 59690 238760

So then delete it.

>>> del(big_list)

But the memory footprint of the process remains very large:

%MEM  RSS  RSS    SZ   VSZ
37.1 190772 190772 47971 191884


Is python leaking memory, or is there something else going on here.



I should note, if I just preallocate the list with 0.0:

>>> big_list = [0.0] * 12000000

and then delete it, I don't see the leak, but then memory footprint is
smaller than you would expect
from 12 million floats:

%MEM  RSS  RSS    SZ   VSZ
 9.4 48700 48700 12453 49812

So I think python is using a shortcut in this case.


I am using Python 2.2.1 on Debian 3.0, kernel 2.4.17-686.

Thanks for the help!