Python memory usage

[David Cournapeau]

On Wed, Oct 29, 2008 at 6:56 PM, petercable at gmail.com
<petercable at gmail.com> wrote:
> On Oct 21, 5:19 pm, Rolf Wester <rolf.wes... at ilt.fraunhofer.de> wrote:
>> Hi,
>>
>> I have the problem that with long running Python scripts (many loops)
>> memory consumption increases until the script crashes. I used the
>> following small script to understand what might happen:
>>
> <snip>
>
> AFAIK, python uses malloc behind the scenes to allocate memory. From
> the malloc man page...
>
> "The  malloc() and free() functions provide a simple, general-purpose
> memory allocation package. The malloc() function returns a pointer to
> a block of at least size bytes suitably aligned for any use. If the
> space assigned by malloc() is overrun, the results are undefined.
>
> The argument to free() is a pointer to a block previously allocated by
> malloc(), calloc(), or realloc(). After free() is executed, this space
> is made available for further  allocation by the application, though
> not returned to the system. Memory is returned to the system only
> upon  termination of  the  application.  If ptr is a null pointer, no
> action occurs. If a random number is passed to free(), the results are
> undefined."

Depending on your malloc implementation, that may not be true. IN
particular, with glibc, bit allocations are done with mmap, and those
areas are unmaped when free is called; any such area is immediatly
returned to the system

http://www.gnu.org/software/libtool/manual/libc/Malloc-Tunable-Parameters.html#Malloc-Tunable-Parameters

David