Reliably call code after object no longer exists or is "unreachable"?

[Jack Bates]

In Python, how can you reliably call code - but wait until an object no
longer exists or is "unreachable"?

I want to ensure that some code is called (excluding some exotic
situations like when the program is killed by a signal not handled by
Python) but can't call it immediately. I want to wait until there are no
references to an object - or the only references to the object are from
unreachable reference cycles


#!/usr/bin/env python

class Goodbye:
  def __del__(self):
    print 'Goodbye, world!'

ref = Goodbye()


$ ./goodbye
Goodbye, world!
$ 


Python's __del__ or destructor method works (above) - but only in the
absence of reference cycles (below). An object, with a __del__ method,
in a reference cycle, causes all objects in the cycle to be
"uncollectable". This can cause memory leaks and because the object is
never collected, its __del__ method is never called


> Circular references which are garbage are detected when the option
> cycle detector is enabled (it's on by default), but can only be
> cleaned up if there are no Python-level __del__() methods involved.


#!/usr/bin/env python

class Goodbye:
  def __del__(self):
    print 'Goodbye, world!'

class Cycle:
  def __init__(self, cycle):
    self.next = cycle
    cycle.next = self

Cycle(Goodbye())


$ ./cycle
$ 


In PEP 342 I read that an object, with a __del__ method, referenced by a
cycle but not itself participating in the cycle, doesn't cause objects
to be uncollectable. If the cycle is "collectable" then when it's
eventually collected by the garbage collector, the __del__ method is
called


> If the generator object participates in a cycle, g.__del__() may not
> be called. This is the behavior of CPython's current garbage
> collector. The reason for the restriction is that the GC code needs to
> "break" a cycle at an arbitrary point in order to collect it, and from
> then on no Python code should be allowed to see the objects that
> formed the cycle, as they may be in an invalid state. Objects "hanging
> off" a cycle are not subject to this restriction.


#!/usr/bin/env python

import sys

class Destruct:
  def __init__(self, callback):
    self.__del__ = callback

class Goodbye:
  def __init__(self):
    self.destruct = Destruct(lambda: sys.stdout.write('Goodbye, world!\n'))

class Cycle:
  def __init__(self, cycle):
    self.next = cycle
    cycle.next = self

Cycle(Goodbye())


$ ./dangle
Goodbye, world!
$ 


However it's *extremely* tricky to ensure that the object with a __del__
method doesn't participate in a cycle, e.g. in the example below, the
__del__ method is never called - I suspect because the object with a
__del__ method is reachable from the global scope, and this forms a
cycle with a frame's f_globals reference? "storing a generator object in
a global variable creates a cycle via the generator frame's f_globals
pointer"


#!/usr/bin/env python

import sys

class Destruct:
  def __init__(self, callback):
    self.__del__ = callback

class Goodbye:
  def __init__(self):
    self.destruct = Destruct(lambda: sys.stdout.write('Goodbye, world!\n'))

class Cycle:
  def __init__(self, cycle):
    self.next = cycle
    cycle.next = self

ref = Cycle(Goodbye())


$ ./global
$ 


Faced with the real potential for reference cycles, how can you reliably
call code - but wait until an object no longer exists or is
"unreachable"?