"Help needed - I don't understand how Python manages memory"

[Steve Holden]

Hank @ITGroup wrote:
> Steve Holden wrote:
>> You are suffering from a pathological condition yourself: the desire 
>> to optimize performance in an area where you do not have any problems. 
>> I would suggest you just enjoy using Python and then start to ask 
>> these questions again when you have a real issue that's stopping you 
>> from getting real work done.
>>
>> regards
>>  Steve
>>
> Hi, Steve,
> This not simply a pathological condition. My people are keeping trying 
> many ways to have job done, and the memory problem became the focus we 
> are paying attention on at this moment.
> Could you please give us some clear clues to obviously call python to 
> free memory. We want to control its gc operation handily as we were 
> using J**A.

Well, now you've told us a little more about your application I can 
understand that you need to be careful with memory allocation.

The best thing you can do is to ensure that your program is reasonably 
decomposed into functions. That way the local namespaces have limited 
lifetimes, and only the values that they return are injected into the 
environment. You also need to be careful in exception processing that 
you do not cause a reference to the stack frame to be retained, as that 
can be a fruitful source of references to objects, rendering them 
non-collectable.

You appear to be stressing the limits of a single program under 
present-day memory constraints. I am afraid that no matter how carefully 
you manage object references, any difference you can make is likely to 
be lost in the noise as far as memory utilization is concerned, and you 
may have to consider using less direct methods of processing your data sets.

The gc module does give you some control over the garbage collector, but 
generally speaking most programs don't even need that much control.

regards
  Steve
-- 
Steve Holden        +1 571 484 6266   +1 800 494 3119
Holden Web LLC              http://www.holdenweb.com/