[issue42969] pthread_exit & PyThread_exit_thread from PyEval_RestoreThread etc. are harmful

[Jeremy Maitin-Shepard]

Jeremy Maitin-Shepard <jeremy at jeremyms.com> added the comment:

Another possible resolution would to simply make threads that attempt to acquire the GIL after Python starts to finalize hang (i.e. sleep until the process exits).  Since the GIL can never be acquired again, this is in some sense the simplest way to fulfill the contract.  This also ensures that any data stored on the thread call stack and referenced from another thread remains valid.  As long as nothing on the main thread blocks waiting for one of these hung threads, there won't be deadlock.


I have a case right now where a background thread (created from C++, which is similar to a daemon Python thread) acquires the GIL, and calls "call_soon_threadsafe" on an asycnio event loop.  I think that causes some Python code internally to release the GIL at some point, after triggering some code to run on the main thread which happens to cause the program to exit.  While `Py_FinalizeEx` is running, the call to "call_soon_threadsafe" completes on the background thread, attempts to re-acquire the GIL, which triggers a call to pthread_exit.  That unwinds the C++ stack, which results in a call to Py_DECREF without the GIL held, leading to a crash.

----------
nosy: +jbms

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42969>
_______________________________________