[C++-sig] Thread safety

[Niall Douglas]

On 18 Dec 2005 at 21:15, Patrick Hartling wrote:

> The only thing I did for thread safety was to add the PyGILState calls to the virtual function wrapper around the call<>.  After all that, my question is, is that suffi> 
> I have a very similar case with my use of Boost.Python. With my code,
> a Python object will be instantiated and then handed off to the
> multi-threaded C++ library. The Python interpreter runs in the
> primordial thread, but the method invocations on the Python object are
> made from a different thread.
> 
> My locking of the GIL has thus far been done before the call to
> boost::python::get_override() or any other Boost.Python functions. I
> changed my code so that the GIL is locked after the call to
> boost::python::get_override() as you have done above. I found that my
> code crashed at the first attempt of the C++ library's attempt to
> invoke a method of the Python object from a spawned thread. The crash
> occurred as a result of the call to boost::python::get_override()
> since it calls into the Python interpreter without the GIL being
> locked. I can provide a stack trace (from GDB) if you would like to
> see it.

You will find this very tricky to get right, as BPL dives into Python 
at weird points such as the container indexing suite iterators. Far 
better to patch BPL to lock and unlock the python GIL around 
entrances into C++ code.

The mechanics of this are not easy. Best download BoostPatches.zip 
from http://svn.berlios.de/viewcvs/tnfox/trunk/Python/ and see the 
changes for yourself. Note that these do not support the modern 
get_override() interface yet (as pyste still uses the old one).

Cheers,
Niall