[New-bugs-announce] [issue20891] PyGILState_Ensure on non-Python thread causes fatal error

[Steve Dower]

New submission from Steve Dower:

In Python 3.4rc3, calling PyGILState_Ensure() from a thread that was not created by Python and without any calls to PyEval_InitThreads() will cause a fatal exit:

Fatal Python error: take_gil: NULL tstate

I believe this was added in http://hg.python.org/cpython/rev/dc4e805ec68a. The attached file has a minimal repro (for Windows, sorry, but someone familiar with low-level *nix threading APIs should be able to translate it easily).

Basically, the sequence looks like this:

1. main() calls Py_Initialize()
2. main() starts new C-level thread
3. main() calls Py_BEGIN_ALLOW_THREADS and waits for the new thread
4. New thread calls PyGILState_Ensure()
5. PyGILState_Ensure() sees that there is no thread state and calls PyEval_InitThreads() (this is what changed in the linked changeset)
6. PyEval_InitThreads() creates the GIL
7. PyEval_InitThreads() calls take_gil(PyThreadState_GET()) (this is what aborts - there is still no thread state)
8. (expected) PyGILState_Ensure() calls PyThreadState_New() and PyEval_RestoreThread().

The attached repro has two workarounds (commented out). The first is to call PyThreadState_New()/PyEval_RestoreThread() from the new thread, and the second is to call PyEval_InitThreads() from the main thread.

The latter is inappropriate for my use, since I am writing a debugger that can inject itself into a running CPython process - there is no way to require that PyEval_InitThreads() has been called and I can't reliably inject it onto the main thread. The first workaround is okay, but I'd rather not have to special-case 3.4 by rewriting PyGILState_Ensure() so that I can safely call PyGILState_Ensure().

----------
components: Extension Modules
files: test.c
messages: 213160
nosy: haypo, pitrou, steve.dower
priority: normal
severity: normal
status: open
title: PyGILState_Ensure on non-Python thread causes fatal error
versions: Python 3.4
Added file: http://bugs.python.org/file34358/test.c

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20891>
_______________________________________