[Python-ideas] Why not break cycles with one __del__?

[Nick Coghlan]

On Tue, Sep 14, 2010 at 3:25 AM, Tim Peters <tim.peters at gmail.com> wrote:
> [Jim Jewett]
>>> The last time I checked ...
>>> single-__del__ cycles were already handled OK.
>
> [Antoine Pitrou]
>> They aren't: ...
>
> Antoine's right, unless things have changed dramatically since last
> time I was intimate with that code.  CPython's "cyclic garbage
> detection" makes no attempt to analyze cycle structure.  It infers
> that all trash it sees must be in cycles simply because the trash
> hasn't already been collected by the regular refcount-based gc.  The
> presence of __del__ on a trash object then disqualifies it from
> further analysis, but there's no analysis of cycle structure
> regardless.

I had a skim through that code last night, and as far as I can tell it
still works that way. However, it should be noted that the cyclic GC
actually does release everything *else* in the cycle - it's solely the
objects with __del__ methods that remain alive.

There does appear to a *little* bit of structural analysis going on -
it looks like the "finalizers" list ends up containing both objects
with __del__ methods, as well as all other objects in the cyclic trash
that are reachable from the objects with __del__ methods.

> Of course it doesn't _have_ to be that way.  Nobody cared enough yet
> to add a pile of new code to special-case cycles with a single
> __del__.

Just from skimming the code, I wonder if, once finalizers has been
figured out, the GC could further partition that list into "to_delete"
(no __del__ method), "to_finalize" (__del__ method, but all referrers
in cycle have no __del__ method) and "uncollectable" (multiple __del__
methods in cycle). Alternatively, when building finalizers, build two
lists: one for objects with __del__ methods and one for objects that
are reachable from objects with __del__ methods. Objects that appear
only in the first list could safely have their finalisers invoked,
while those that also in the latter could not.

This is definitely a case of "code talks" though - there's no
fundamental problem with the idea, but also no great incentive for
anyone to code it when __del__ is comparatively easy to avoid
(although not trivial, see Raymond's recent modifications to
OrderedDictionary to avoid exactly this issue).

Or, accept that __del__ is evil, and try to come up with a workable
proposal for that better weakref callback based scheme Jim mentioned.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia