[New-bugs-announce] [issue44422] threading.enumerate(): reentrant call during a GC collection hangs
STINNER Victor
report at bugs.python.org
Mon Jun 14 17:26:28 EDT 2021
New submission from STINNER Victor <vstinner at python.org>:
The threading.enumerate() code is simple:
---
# Active thread administration
_active_limbo_lock = _allocate_lock()
_active = {} # maps thread id to Thread object
_limbo = {}
def enumerate():
with _active_limbo_lock:
return list(_active.values()) + list(_limbo.values())
---
values(), list() and list+list operations can call trigger a GC collection depending on the GC thresholds, created Python objects, etc.
During a GC collection, a Python object destructor (__del__) can again call threading.enumerate().
Problem: _active_limbo_lock is not re-entrant lock and so the second call hangs.
In practice, this issue was seen in OpenStack which uses eventlet, when a destructor uses the logging module. The logging module uses threading.current_thread(), but this function is monkey-patched by the eventlet module.
Extract of the eventlet function:
---
def current_thread():
...
found = [th for th in __patched_enumerate() if th.ident == g_id]
...
---
https://github.com/eventlet/eventlet/blob/master/eventlet/green/threading.py
Attached PR makes _active_limbo_lock re-entrant to avoid this issue. I'm not sure if it's ok or not, I would appreciate to get a review and your feedback ;-)
Attached file triggers the threading.enumerate() hang on re-entrant call.
----------
components: Library (Lib)
files: rec_threading.py
messages: 395851
nosy: vstinner
priority: normal
severity: normal
status: open
title: threading.enumerate(): reentrant call during a GC collection hangs
versions: Python 3.10, Python 3.11, Python 3.9
Added file: https://bugs.python.org/file50110/rec_threading.py
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44422>
_______________________________________
More information about the New-bugs-announce
mailing list