Python not giving free memory back to the os get's me in real problems ...

[Grant Edwards]

On 2007-04-25, Steven Howe <howe.steven at gmail.com> wrote:

>> I'm a bit fuzzy on this, but I don't think there _is_ a
>> practical way to "return memory to the OS" in many OSes.  For
>> example in Unix the C library uses the sbrk() call to increase
>> the size of the data segment when additional memory is needed
>> to handle soemthing like malloc() calls.  
>>
>> In theory, one can use brk() to reduce the size of the data
>> segment, but I think making the segment smaller will produce
>> undefined behavior if any of the standard C library's dynamic
>> memory routines (e.g. malloc/free) have ever been used by the
>> program prior to the call to brk().
>
> Interesting questions. What happens when an object is
> 'cleaned' up by using the 'del' command. Does the memory space
> stay in the python process, get handed back to the OS, or some
> combination of both?

Assuming the python interpreter free()s the memory, my
understanding is that on Unixes the memory is returned to the
pool used by malloc(), but is not returned to the OS since
there isn't a practical way to ensure that the memory at the
"end" of the data segment is not used.

In theory, you could walk the data structures used by
free/malloc and try to figure out if a free() should allow
brk() to be called to reduce the data segment size.  That would
only happen if the free() call was freeing data that was at the
"end" of the data segment.  It's possible that some standard C
libraries do that, but most of what I remember reading implies
that they don't. I don't really keep current on libc details,
so my info might be way out of date.

Asking on the gnu libc mailing list would probably provide a
more authoritative answer.  I'd wager that they even know what
other non-Gnu libc implementations do.

> I remember 'C' on VMS at least, could be coerced into return
> memory on block boundaries.

You can call brk() in a C program to reduce the size of the
data segment, but that may result in free and malloc breakage
because you've returned (behind their back) some of the memory
they are managing.

> 'C++' was suppose to have garbage collect, but I was always
> doubtful it worked well.

-- 
Grant Edwards                   grante             Yow! Why is everything made
                                  at               of Lycra Spandex?
                               visi.com