Embedded/Extended w/C++ - Thread error

[Gordon McMillan]

Mary Austin writes:

> Hi!  I'm attempting to have python call functions in C++ and have
> methods in C++ call functions in Python (all from the same C++ dll).
>  The first direction, I have working just fine (Python calling C++
> functions). However, the other direction is not so easy.  Whenever I
> attempt to call PyImport_ImportModule("<insert any module here>"),
> it bombs completely - abnormal program termination.  

> The error I get just before python.exe dies is "Fatal Python error:
> PyThreadState_Get: no current thread". 

> I'm creating threads from within C++ (the code creates a thread for
> a socket to accept connections as well as a thread for each client
> that's connected), but Python shouldn't care about them, as the C++
> threads are created by another class than what's calling all the
> Python commands. (At least I don't think python should care about
> them.)

> Anyone have any idea why PyImport_ImportModule, PyEval_CallObject,
> and PyObject_CallFunction would cause a thread error when I'm not
> using Python threads (only C++ threads from within classes that
> don't talk to Python)?

There's a couple possibilities here. The most likely is that, in 
fact, you are calling into Python from a different (C++) thread 
than the thread that created the Python interpreter. Since your code 
is all in a dll, you might not be aware of this - it might be a 
thread created in the .exe.

You say that the "extending" direction works, which means that you
got one C++ to Python call working. The thread that made that call is 
the _only_ thread the can safely make further calls without creating 
a Python thread state.

This is a very hairy area of the API, which was recently discussed on 
the Thread SIG. You should probably download the archive for April 
when this problem was discussed in (gory) detail.

As a general outline, you will need to capture the interpreter state 
once you've started Python. Then you'll need to use that to create 
new thread states for each C++ thread. In the archive, you'll find 
some pointers to working code.

Your other possibility is to marshall everything to the one and only 
Python-running thread. Since you're in a dll, that probably means 
doing TLS, or otherwise managing thread ids yourself. Which is 
probably equally as hairy, and won't take advantage of Python's 
ability to multithread.

- Gordon