[Python-Dev] Design question: call __del__ for cyclical garbage?

[Tim Peters]

[Guido]
> ...
> Someone (Tim?) in the past suggested a different solution (probably
> found in another language): for objects that are collected as part of
> a cycle, the destructor isn't called at all.  The memory is freed
> (since it's no longer reachable), but the destructor is not called --
> it is as if the object lives on forever.

Stroustrup has written in favor of this for C++.  It's exactly the kind of
overly slick "good argument" he would never accept from anyone else <0.1
wink>.

> This is theoretically superior, but not practical: when I have an
> object that creates a temp file, I want to be able to reliably delete
> the temp file in my destructor, even when I'm part of a cycle!

A member of the C++ committee assured me Stroustrup is overwhelmingly
opposed on this.  I don't even agree it's theoretically superior:  it relies
on the fiction that gc "may never occur", and that's just silly in practice.

You're moving down the Java path.  I can't possibly do a better job of
explaining the Java rules than the Java Language Spec. does for itself.  So
pick that up and study section 12.6 (Finalization of Class Instances).  The
end result makes little sense to users, but is sufficient to guarantee that
Java itself never blows up.

Note, though, that there is NO good answer to finalizers in cycles!  The
implementation cannot be made smart enough to both avoid trouble and "do the
right thing" from the programmer's POV, because the latter is unknowable.
Somebody has to lose, one way or another.

Rather than risk doing a wrong thing, the BDW collector lets cycles with
finalizers leak.  But it also has optional hacks to support exceptions for
use with C++ (which sometimes creates self-cycles) and Java.  See

    http://reality.sgi.com/boehm_mti/finalization.html

for Boehm's best concentrated <wink> thoughts on the subject.

The only principled approach I know of comes out of the Scheme world.
Scheme has no finalizers, of course.  But it does have gc, and the concept
of "guardians" was invented to address all gc finalization problems in one
stroke.  It's extremely Scheme-like in providing a perfectly general
mechanism with no policy whatsoever.  You (the Scheme programmer) can create
guardian objects, and "register" other objects with a guardian.  At any
time, you can ask a guardian whether some object registered with it is
"ready to die" (i.e., the only thing keeping it alive is its registration
with the guardian).  If so, you can ask it to give you one.  Everything else
is up to you:  if you want to run a finalizer, your problem.  If there are
cycles, also your problem.  Even if there are simple non-cyclic
dependencies, your problem.  Etc.

So those are the extremes:  BDW avoids blame by refusing to do anything.
Java avoids blame by exposing an impossibly baroque implementation-driven
finalization model.  Scheme avoids blame by refusing to do anything "by
magic", but helps you to shoot yourself with the weapon of your choice.

That bad news is that I don't know of a scheme *not* at an extreme!

It's extremely un-Pythonic to let things leak (despite that it has let
things leak for a decade <wink>), but also extremely un-Pythonic to make
some wild-ass guess.

So here's what I'd consider doing:  explicit is better than implicit, and in
the face of ambiguity refuse the temptation to guess.  If a trash cycle
contains a finalizer (my, but that has to be rare. in practice, in
well-designed code!), don't guess, but make it available to the user.  A
gc.guardian() call could expose such beasts, or perhaps a callback could be
registered, invoked when gc finds one of these things.  Anyone crazy enough
to create cyclic trash with finalizers then has to take responsibility for
breaking the cycle themself.  This puts the burden on the person creating
the problem, and they can solve it in the way most appropriate to *their*
specific needs.  IOW, the only people who lose under this scheme are the
ones begging to lose, and their "loss" consists of taking responsibility.

when-a-problem-is-impossible-to-solve-favor-sanity<wink>-ly y'rs  - tim