[Python-Dev] Problems with the Python Memory Manager

[JustFillBug]

On 2005-11-16, Travis Oliphant <oliphant at ee.byu.edu> wrote:
> Josiah Carlson wrote:
>>I seemed to have misunderstood the discussion.  Was the original user
>>accessing and saving copies of many millions of these doubles?  
>>
> He *was* accessing them (therefore generating a call to an array-scalar 
> object creation function).  But they *weren't being* saved.  They were 
> being deleted soon after access.   That's why it was so confusing that 
> his memory usage should continue to grow and grow so terribly.
>
> As verified by removing usage of the Python PyObject_MALLOC function, it 
> was the Python memory manager that was performing poorly.   Even though 
> the array-scalar objects were deleted, the memory manager would not 
> re-use their memory for later object creation. Instead, the memory 
> manager kept allocating new arenas to cover the load (when it should 
> have been able to re-use the old memory that had been freed by the 
> deleted objects--- again, I don't know enough about the memory manager 
> to say why this happened).

Well, the user have to call garbage collection before the memory were
freed. Python won't free memory when it can allocate more. It sucks but
it is my experience with python. I mean when python start doing swap on
my machine, I have to add manual garbage collection calls into my codes.