[issue36402] test_threading: test_threads_join_2() failed with "Fatal Python error: Py_EndInterpreter: not the last thread" on AMD64 FreeBSD CURRENT Shared 3.x

[STINNER Victor]

STINNER Victor <vstinner at redhat.com> added the comment:

threading._shutdown() uses threading.enumerate() which iterations on threading._active.

threading.Thread registers itself into threading._active using its _bootstrap_inner() method. It unregisters itself when _bootstrap_inner() completes, whereas its is_alive() method still returns true: since the underlying native thread still runs and the Python thread state still exists.

_thread._set_sentinel() creates a lock and registers a tstate->on_delete callback to release this lock. It's called by threading.Thread._set_tstate_lock() to set threading.Thread._tstate_lock. This lock is used by threading.Thread.join() to wait until the thread completes.


_thread.start_new_thread() calls the C function t_bootstrap() which ends with:

    tstate->interp->num_threads--;
    PyThreadState_Clear(tstate);
    PyThreadState_DeleteCurrent();
    PyThread_exit_thread();

_PyThreadState_DeleteCurrent() calls tstate->on_delete() which releases threading.Thread._tstate_lock lock.

In test_threads_join_2() test, PyThreadState_Clear() blocks on clearing thread variables: the Sleeper destructor of the Sleeper instance sleeps.

The race condition is that:

* threading._shutdown() rely on threading._alive
* Py_EndInterpreter() rely on the interpreter linked list of Python thread states: interp->tstate_head.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36402>
_______________________________________