[Python-Dev] refleaks & test_tcl & threads

[Thomas Wouters]

On 4/16/06, Thomas Wouters <thomas at python.org> wrote:

> test_threading_local is not entirely consistent, but it looks a lot more
> reliable on my box than on Neal's automated mails:
>
> test_threading_local
> beginning 11 repetitions
> 12345678901
> ...........
> test_threading_local leaked [34, 34, 34, 34, 34, 26, 26, 22, 34]
> references
>

This is caused by _threading_local.local's __del__ method, or rather the
fact that it's part of a closure enclosing threading.enumerate. Fixing the
inner __del__ to call enumerate (which is 'threading.enumerate') directly,
rather than through the cellvar 'threading_enumerate', makes the leak go
away. The fact that the leakage is inconsistent is actually easily
explained: the 'local' instances are stored on the 'currentThread' object
indexed by 'key', and keys sometimes get reused (since they're basically
id(self)), so sometimes an old reference is overwritten. It doesn't quite
explain why using the cellvar causes the cycle, nor does it explain why
gc.garbage remains empty. I guess some Thread objects linger in threading's
_active or _limbo dicts, but I have no idea why having a cellvar in the
cycle matters; they seem to be participating in GC just fine, and I cannot
reproduce the leak with a simpler test.

And on top of that, I'm not sure *why* _threading_local.local is doing the
song and dance to get a cellvar. If the global 'enumerate' (which is
threading.enumerate) disappears, it will be because Python is cleaning up.
Even if we had a need to clean up the local dict at that time (which I don't
believe we do), using a cellvar doesn't guarantee anything more than using a
global name. Chances are very good that the 'threading' module has also been
cleared, meaning that while we still have a reference to threading.enumerate,
it cannot use the three globals it uses (_active, _limbo,
_active_limbo_lock.) All in all, I think matters improve significantly if it
just deals with the NameError it'll get at cleanup (which it already does.)

--
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060416/24be3ae6/attachment-0001.htm