The OOM-Killer vs. Python

[Martin v. Loewis]

gerson.kurz at t-online.de (Gerson Kurz) writes:

> Anyway, even though I believe that this is more of a fault of the
> Linux Kernel VM quality than the script (the system has been running
> fine for months and now two kills in one week - that smells fishy to
> me) - how CAN I controll the memory usage for pythons GC? Is there a
> way to easily find out in python which memory is allocated at runtime,
> and by which objects?

If you suspect that you have cyclic garbage, turn on gc.DEBUG_SAVEALL,
and inspect gc.garbage. Before doing so, you should inspect gc.garbage
alone - it ought to be empty in a long-running program under all
cicrcumstances.

If you find objects on gc.garbage, you may try to break cycles
explicitly when you know you don't need the objects anymore - people
take pride in having their long-running scripts work even without gc.

If you don't create any cyclic garbage, you can find all container
objects with gc.get_objects.

If you find that gc.get_objects does not grow longer over time, but
your process still consumes more memory, one of your C extensions has
a refcounting bug.

HTH,
Martin