[Python-Dev] reference leaks, __del__, and annotations

[Thomas Wouters]

On 3/31/06, Jim Jewett <jimjjewett at gmail.com> wrote:
>
> The checkins list has been struggling with generator reference leaks;
> the latest conclusion was that some are unavoidable because of __del__
> cycles.


That was Tim's conclusion, but I wasn't quite done thinking about it ;)


> That sort of defeats the purpose of resource managers.  (Yes,
> it can be worked around on a case-by-case basis.)
>
> As I see it, part of the problem is that
>
> (1)  When there is a cycle, python refuses to guess.
> (2)  There is no way for a __del__ method to hint at ordering constraints.
> (3)  There is no lightweight version of __del__ to say "I don't care
> about ordering constraints."


An additional (and much more complicating) problem is that __del__ can (and
is allowed to) revive 'self'. That means that halfway through your cleanup,
however you decided to do it, you can suddenly find out that you shouldn't
be cleaning up at all. And of course any given __del__ can rely on all of
the parts of the cycle, even if one of those parts _claims_ it can safely
break the cycle.

I think there are three different scenarios involving cycles and/or __del__
methods:

 - An object may conciously create a cycle, and know how to resolve it. A
'__breakcycle__' method or such may be the right way to handle those cases.
It would have to be pretty sure that no one outside itself can (or should)
rely on the attribute or closure it breaks the cycle with, though. (Sensible
exceptions ought to be fine, IMHO.)

 - An object may not care about cycles, but want to do some cleanup when it
is deleted. The C types have 'tp_dealloc' for this, and that's what PyFile
uses to close files. If you want to emulate this behaviour in Python, you
are forced to use __del__, creating the unreclaimable cycle problem. Perhaps
a __dealloc__ class/staticmethod makes sense; it would be passed a
dictionary of instancedata, but not 'self', so it can never revive 'self'
and can be sanely used in cycles -- that is, some of its instancedata may
still suddenly be None, but that's a problem with __del__ methods and global
variables, too.

 - An object may care about cycles, actually need a __del__ method that can
revive the object, but not have a sane way to say 'break the cycle at this
point'. Generators may be an example of that kind of object, although I'm
not sure: they could throw() an exception to end the cycle. I don't think we
can reclaim cycles where none of the objects can fairly break the cycle.

--
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060331/624508fa/attachment.html