C-extension not threadable?

[Martin v. Loewis]

Dave Cole <djc at object-craft.com.au> writes:

> Martin> Yes, the Global Interpreter Lock (GIL). When C code runs that
> Martin> was invoked from Python code, it holds the GIL, and only one
> Martin> such C code can run at any time. So you need to release the
> Martin> lock, using Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS.
> 
> If the API which is being wrapped by the extension module utilises
> callbacks, and you want to make those callbacks available to Python
> code, and you want to do this in a threading environment where the GIL
> is released when calling the API, then the above will not work.

You must not call *any* Python API while not holding the GIL, yes.

That doesn't mean that my approach won't work - it is just not the
full story.

> This seems to be working fine so far (fingers crossed).

Sounds good; if possible, you should indeed pass the thread state
forward to the callback. If you have a lock around the library,
passing that thread state in a global variable works fine (see
_tkinter for an example).

If callbacks can come out of arbitrary threads, or if it is not
possible to pass the thread-state through, you can always create a new
thread state when entering the callback. In that case, you still need
to obtain the interpreter somehow.

Regards,
Martin