[Python-Dev] Design question: call __del__ only after successful __init__?

Tim Peters tim_one@email.msn.com
Sat, 4 Mar 2000 23:11:12 -0500


[Guido]
> OK, so we're down to this one point: if __del__ resurrects the object,
> should __del__ be called again later?  Additionally, should
> resurrection be made illegal?

I give up on the latter, so it really is just one.

> I can easily see how __del__ could *accidentally* resurrect the object
> as part of its normal cleanup ...
> In this example, the helper routine will eventually delete the object
> from its cache, at which point it is truly deleted.  It would be
> harmful, not helpful, if __del__ was called again at this point.

If this is something that happens easily, and current behavior is harmful,
don't you think someone would have complained about it by now?  That is,
__del__ *is* "called again at this point" now, and has been for years &
years.  And if it happens easily, it *is* happening now, and in an unknown
amount of existing code.  (BTW, I doubt it happens at all <wink> -- people
tend to write very simple __del__ methods, so far as I've ever seen)

> Now, it is true that the current docs for __del__ imply that
> resurrection is possible.

"imply" is too weak.  The Reference Manual's "3.3.1 Basic customization"
flat-out says it's possible ("though not recommended").  The precise meaning
of the word "may" in the following sentence is open to debate, though.

> The intention of that note was to warn __del__ writers that in the case
> of accidental resurrection

Sorry, but I can't buy this:  saying that *accidents* are "not recommended"
is just too much of a stretch <wink/frown>.

> __del__ might be called again.

That's a plausible reading of the following "may", but not the only one.  I
believe it's the one you intended, but it's not the meaning I took prior to
this.

> The intention certainly wasn't to allow or encourage intentional
resurrection.

Well, I think it plainly says it's supported ("though not recommended").  I
used it intentionally at KSR, and even recommended it on c.l.py in the dim
past (in one of those "dark & useless" threads <wink>).

> Would there really be someone out there who uses *intentional*
> resurrection?  I severely doubt it.  I've never heard of this.

Why would anyone tell you about something that *works*?!  You rarely hear
the good stuff, you know.  I gave the typical pattern in the preceding msg.
To flesh out the motivation more, you have some external resource that's
very expensive to set up (in KSR's case, it was an IPC connection to a
remote machine).  Rights to use that resource are handed out in the form of
an object.  When a client is done using the resource, they *should*
explicitly use the object's .release() method, but you can't rely on that.
So the object's __del__ method looks like (for example):

def __del__(self):

    # Code not shown to figure out whether to disconnect:  the downside to
    # disconnecting is that it can cost a bundle to create a new connection.
    # If the whole app is shutting down, then of course we want to
disconnect.
    # Or if a timestamp trace shows that we haven't been making good use of
    # all the open connections lately, we may want to disconnect too.

    if decided_to_disconnect:
        self.external_resource.disconnect()
    else:
        # keep the connection alive for reuse
        global_available_connection_objects.append(self)

This is simple & effective, and it relies on both intentional resurrection
and __del__ getting called repeatedly.  I don't claim there's no other way
to write it, just that there's *been* no problem doing this for a millennium
<wink>.

Note that MAL spontaneously sketched similar examples, although I can't say
whether he's actually done stuff like this.


Going back up a level, in another msg you finally admitted <wink> that you
want "__del__ called only once" for the same reason Java wants it:  because
gc has no idea what to do when faced with finalizers in a trash cycle, and
settles for an unprincipled scheme whose primary virtue is that "it doesn't
blow up" -- and "__del__ called only once" happens to be convenient for that
scheme.

But toss such cycles back to the user to deal with at the Python level, and
all those problems go away (along with the artificial need to change
__del__).  The user can break the cycles in an order that makes sense to the
app (or they can let 'em leak!  up to them).

    >>> print gc.get_cycle.__doc__
    Return a list of objects comprising a single garbage cycle; [] if none.

    At least one of the objects has a finalizer, so Python can't determine
the
    intended order of destruction.  If you don't break the cycle, Python
will
    neither run any finalizers for the contained objects nor reclaim their
    memory.  If you do break the cycle, and dispose of the list, Python will
    follow its normal reference-counting rules for running finalizers and
    reclaiming memory.

That this "won't blow up" either is just the least of its virtues <wink>.

you-break-it-you-buy-it-ly y'rs  - tim