[issue15751] Support subinterpreters in the GIL state API

[Nick Coghlan]

Nick Coghlan added the comment:

Thinking about it, I believe there still needs to be a concept of an "active thread state" TLS key in order to deal with Graham's original problem. Specifically, if PyGILState_EnsureEx is used to associate the thread with a particular interpreter, then subsequent calls to PyGILState_Ensure from *that thread* should get the explicitly associated interpreter, rather than the main interpreter.

Then, the lookup sequence for "interpreter=NULL" would be:

1. Check the active TLS key, if set, use that thread state
2. If not set, look up the main interpreter's TLS key. If set, also set that on the active TLS key and use that thread state.
3. If still not set, create a new thread state on the main interpreter, set it for both the active TLS and the main interpreter's TLS key and use that thread state

A similar approach almost works when requesting a specific interpreter, but where that goes wrong is when the active TLS key is *already set*. You can't just overwrite it, because that will cause problems for subsequent PyGIL_Release calls. You could just make it an error, but I think Graham's original idea makes it possible to do better than that.

Specifically, a PyGILState_SwitchInterpreter API could focus solely on the manipulation of the "active thread state" TLS key entry. The sequence of commands in mod_wsgi would then look like:

old_interp = PyGILState_SwitchInterpreter(target_interp);
old_gil = PyGILState_Ensure();
/* Call into Python using target_interp */
PyGILState_Release(old_gil);
PyGILState_SwitchInterpreter(old_interp); /* May not be needed in the mod_wsgi case, since it knows it is making the outermost call into the PyGILState_* APIs */

All of the other elements of Antoine's proposal (i.e. the per-interpreter TLS key entries) would still be needed, it's just that the API for associating a thread with an interpreter would remain separated from that of associating the thread with a particular thread state.

The big advantage of doing it this way is that it will nest properly, whereas PyGILState_EnsureEx would need a more complicated API to correctly report both the old interpreter state and the old GIL state within that interpreter.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15751>
_______________________________________