Endless GIL and thread confusion

[kyosohma at gmail.com]

On Oct 21, 11:25 am, enska <ensifer... at hotmail.com> wrote:
> Can someone clarify the steps needed to make access to the interpreter
> safe from multiple threads?
>
> I've been reading the docs for days and I still find them very confusing
> and misleading. For example does the PyGILState_Ensure() function lock the
> GIL or just create the thread state? Is thread supposed to call it once on
> creation and then the release function before exiting or is this the
> preferred mechanism to handle the locking? Most of the confusion stems
> from this segment in the documentation.
>
> "...when threads are created from C, they don't have the global
> interpreter lock, nor is there a thread state data structure for them.
> Such threads must bootstrap themselves into existence, by first creating a
> thread state data structure, then acquiring the lock, and finally storing
> their thread state pointer, before they can start using the Python/C API.
> When they are done, they should reset the thread state pointer, release
> the lock, and finally free their thread state data structure.
>
> Beginning with version 2.3, threads can now take advantage of the
> PyGILState_*() functions to do all of the above automatically."
>
> Anyway, currently my code looks like this:
>
> void foobar(...)
> {
>     PyGILState state = PyGILState_Ensure();
>
>     // make python calls
>
>     PyGILState_Release(state);
>
> }
>
> void blablah(...)
> {
>     PyGILState state = PyGILState_Ensure();
>
>     // make python calls
>
>     PyGILState_Release(state);
>
> }
>
> My python Initialization code looks like this
>
> void init_python(char* progname, char* bindir)
> {
>     Py_SetProgramName(progname);
>     PyEval_InitThreads();
>     Py_InitializeEx(0);
>
>     char* argv[] = {progname, bindir};
>     PySys_SetArgv(2, argv);
>
> }
>
> calling foobar() or blablah() from the main thread works as expected, but
> if the second thread calls them it locks up in the call to
> PyGILState_Ensure().
>
> I have tried adding a call to PyEval_ReleaseLock() in the init_python
> function to make sure that the main thread is not holding the GIL, but
> this causes the application to segfault when it is calling Py_Finalize()
> so it is clear that this is not correct.
>
> Please advice.
>
> Thanks.

I use the Threading module whenever I need to use threads. It works
quite nicely and I have yet to have any problems with it, except for a
little goofiness with WMI that was explained to me long ago. You might
check that module out.

The wiki for wxPython actually has a good tutorial on threads:

http://wiki.wxpython.org/LongRunningTasks

And here's an article on usurping the GIL:

http://www.pyzine.com/Issue001/Section_Articles/article_ThreadingGlobalInterpreter.html

Hope that helps a little.

Mike