[issue36427] Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado <pablogsal at gmail.com>:

Currently PyEval_RestoreThread and its callers (mainly PyGILState_Ensure) can terminate the thread if the interpreter is finalizing:

PyEval_RestoreThread(PyThreadState *tstate)
{
    if (tstate == NULL)
        Py_FatalError("PyEval_RestoreThread: NULL tstate");
    assert(gil_created());

    int err = errno;
    take_gil(tstate);
    /* _Py_Finalizing is protected by the GIL */
    if (_Py_IsFinalizing() && !_Py_CURRENTLY_FINALIZING(tstate)) {
        drop_gil(tstate);
        PyThread_exit_thread();
        Py_UNREACHABLE();
    }
    errno = err;

    PyThreadState_Swap(tstate);
}

This behaviour that protects against problems due to daemon threads registered with the interpreter can be *very* surprising for C-extensions that are using these functions to implement callbacks that can call into Python. These callbacks threads are not owned by the interpreter and are usually joined by someone else, ending in deadlocks in many situations.

I propose to add a warning to the documentation to inform users about this situation.

----------
components: Interpreter Core
messages: 338826
nosy: eric.snow, pablogsal, pitrou, vstinner
priority: normal
severity: normal
status: open
title: Document that PyEval_RestoreThread and PyGILState_Ensure can terminate the calling thread
versions: Python 3.7, Python 3.8

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue36427>
_______________________________________