[Python-Dev] A surprising case of cyclic trash

[Tim Peters]

This comes (indirectly) from a user of my doctest.py, who noticed that
sometimes tempfiles created by his docstring tests got cleaned up (via
__del__), but other times not.  Here's a hard-won self-contained program
illustrating the true cause:

class Critical:
    count = 0
    def __init__(self):
        Critical.count = Critical.count + 1
        self.id = Critical.count
        print "acquiring Critical", self.id
    def __del__(self):
        print "releasing Critical", self.id

good = "temp = Critical()\n"
bad = "def f(): pass\n" + good

basedict = {"Critical": Critical}

for test in good, bad, good:
    print "\nStarting test case:"
    print test
    exec compile(test, "<string>", "exec") in basedict.copy()

And here's output:

D:\Python>python misc\doccyc.py

Starting test case:
temp = Critical()

acquiring Critical 1
releasing Critical 1

Starting test case:
def f(): pass
temp = Critical()

acquiring Critical 2

Starting test case:
temp = Critical()

acquiring Critical 3
releasing Critical 3

D:\Python>


That is, in the "bad" case, which differs from the "good" case merely in
defining an unreferenced function, temp.__del__ not only doesn't get
executed "when expected", it never gets executed at all.

This appears to be due to a cycle between the function object and the
anonymous dict passed to exec, causing the entire dict to become immortal,
thus making "temp" immortal too.

I can fiddle the doctest framework to manually nuke the temp dict it creates
for execution context; the same kind of leak likely occurs in any exec'ed
string that contains a function defn.

For future reference, note that the finalizer in question belongs to an
object not itself in a cycle, it's an object reachable only from a dead
cycle.

the-users-don't-stand-a-chance<wink>-ly y'rs  - tim