Self healthcheck
Frank Millman
frank at chagford.com
Wed Jan 22 03:56:30 EST 2014
"Asaf Las" <roegltd at gmail.com> wrote in message
news:58c541ab-c6e1-45a8-b03a-8597ed7ecb48 at googlegroups.com...
>
> Yes the question was about CPython. But i am not after CPython leaks
> though detecting these would be good, but my own mistakes leading to
> accumulation of data in mutable structures.
> there will be few processes running python code standalone communicating
> across servers and every activity will be spread over time so
> i have to persistently keep record of activity and remove it later when
> activity is finished.
I had a similar concern. My main worry, which turned out to be well-founded,
was that I would create an object as a result of some user input, but when
the user had finished with it, and in theory it could be garbage-collected,
in practice it would not be due to some obscure circular reference
somewhere.
For short-running tasks this is not a cause for concern, but for a
long-running server these can build up over time and end up causing a
problem.
My solution was to log every time an object was created, with some
self-identifying piece of information, and then log when it was deleted,
with the same identifier. After running the program for a while I could then
analyse the log and ensure that each creation had a corresponding deletion.
The tricky bit was logging the deletion. It is a known gotcha in Python that
you cannot rely on the __del__ method, and indeed it can cause a circular
reference in itself which prevents the object from being garbage-collected.
I found a solution somewhere which explained the use of a 'delwatcher'
class. This is how it works -
class MainObject:
def __init__(self, identifier):
self._del = delwatcher('MainObject', identifier)
class delwatcher:
def __init__(self, obj_type, identifier):
self.obj_type = obj_type
self.identifier = identifier
log('{}: id={} created'.format(self.obj_type, self.identifier))
def __del__(self):
log('{}: id={} deleted'.format(self.obj_type, self.identifier))
In this case calling __del__() is safe, as no reference to the main object
is held.
If you do find that an object is not being deleted, it is then
trial-and-error to find the problem and fix it. It is probably a circular
reference
HTH
Frank Millman
More information about the Python-list
mailing list