Missing stack frames?

[Nikolaus Rath]

dieter <dieter at handshake.de> writes:
[...]
> Someone else already mentioned that the "close" call
> can come from a destructor. Destructors can easily be called
> at not obvious places (e.g. "s = C(); ... x = [s for s in ...]";
> in this example the list comprehension calls the "C" destructor
> which is not obvious when one looks only locally).
> The destructor calls often have intervening C code (which
> one does not see). However, in your case, I do not see
> why the "cgi" module should cause a destructor call of one
> of your server components.

Paul, dieter, you are my heroes. It was indeed an issue with a
destructor. It turns out that the io.RawIOBase destructor calls
self.close(). If the instance of a derived class is part of a reference
cycle, it gets called on the next routine run of the garbage
collector -- with the stack trace originating at whatever statement was
last executed before the gc run.

The following minimal example reproduces the problem:

#!/usr/bin/env python3
import io
import traceback
import threading

class Container:
    pass

class InnocentVictim(io.RawIOBase):
    def close(self):
        print('close called in %s by:'
              % threading.current_thread().name)
        traceback.print_stack()

def busywork():
    numbers = []
    for i in range(500):
        o = Container()
        o.l = numbers
        numbers.append(o)

        if i % 87 == 0:
            numbers = []

l = [ InnocentVictim() ]
l[0].cycle = l
del l

t = threading.Thread(target=busywork)
t.start()
t.join()


If you run this, you could things like:

close called in Thread-1 by:
  File "/usr/lib/python3.4/threading.py", line 888, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "./test.py", line 18, in busywork
    o = Container()
  File "./test.py", line 13, in close
    traceback.print_stack()


Ie, a method being called by a thread that doesn't have access to the
object, and without any reference to the call in the source.


I am left wondering:

 - Is there really a point in the RawIOBase destructor calling close?
 - Is there some way to make the call stack for destructors less confusing?

    
Best,
-Nikolaus

-- 
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F

             »Time flies like an arrow, fruit flies like a Banana.«