[C++-sig] Regarding thread compatibility of boost python

[Andrew Athan]

Could someone please elaborate on what is and is not safe to do with 
respect to threads and boost python?

My specific situation involves an embedded python interpreter within a 
C++ app that uses multiple NPTL threads.

The Python module which is initially called creates a number of threads 
within Python code.

Following initialization, calls tend to originate from C++ into Python, 
and I can guarantee that these are serialized so that no more than one 
thread is calling into Python at once.

However, Python may call back out to C++, and when doing so I need to 
ensure thread safety of the C++ code.  However, such "backout" calls (to 
coin a new term) never result in an intermediate callback into Python.  
I.e., C++->Python->C++ is possible, but not C++->Python->C++->Python.  
And, such "backout" calls are not long running.


Here are what I believe are the correct answers:

*  I assume that thread safety of the C++ code, when invoked from within 
Python, is really a matter of the C++ code itself.  Python threads map 
to NPTL threads, correct?  So, it's possible that my C++ code will be 
invoked from a different thread than the one used to call into Python.  
Therefore, I should appropriately protect the C++ code using NPTL mutexes.

* Calls into Python should be wrapped with PyGILState_Ensure() and 
PyGILState_Release()

* "backout" calls don't have to be protected, really, since they are not 
long running, it's probably safe not to wrap them in 
Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS



Despite the constraints I have described above, which seem (unless 
someone corrects me) to relax the thread safety requirements, I am 
looking into using the boost python patches in the Tn FOX library.  Can 
anyone characterize these?  There is not much included in the sources by 
way of documentation.  I have not yet looked into the actual Tn FOX 
documentation package.

Thanks,
A.