Embeded Multi-Threaded Python: PyEval_InitThread(), PyEval_SaveThread(),...

Nicolas Duchastel nicolas at otelnet.com
Wed Sep 19 14:28:13 EDT 2001


I am trying to use the Python/C API function to get an embeded Interpreter
into a multi-threaded C++ application. We are using Pthreads on Solaris 7.
My application works fine, but only with 1 thread; and then, when I stop it,
it core dumps.
I am using Python 2.1.1 built on Solaris 7 with the SUN's C++ 5.2 compiler.

Here is a simplified architecture of the system:
-----------------------------------------------
I have an C++ application which creates a bunch of threads and then
accepts messages. Each messages is a request to execute some Python code.
Here's some pseudo-code of a 4 threaded system:
   thread A: // INIT thread
       do init stuff;
       init Python interpreter:
           Py_Initializ();
           PyImport_ImportModule("MyModule");
           ...
       start thread B;
       start thread C;
       end this thread;

   thread B and C: // WORKER thread
       wait and receive a new message;
       if (msg is exit_message)
          start Thread D;
          stop other worker thread (B or C);
          end this thread;
       else
          call python code:
             PyRun_SimpleString(...);
            OR
             PyObbject_CallMethod(...);

   thread D: // CLEAN-UP thread
      wait for threads B and C to be stopped;
      clean-up stuff
      Py_Finalize();
       
QUESTIONS:
---------
 1) what are the calls that I should be making !? in what order ?
    any examples ? (i.e. not example of Python code executing some
    multi-threaded stuff using the thread module, but rather some C/C++
    code using the API C functions to implement a multi-threaded Interpreter).

 2) what is the init sequence ? i.e. should PyEval_InitThreads be called ?
    if so, before or after Py_Initialized() ? What objects need to be created ?
    How ? (e.g. how do you create the locks ? what about the Thread State ?)

 3) does the Python interpreter remember which OS thread is running what ?
    i.e. one of my hypothesis for explaining the core dump is that the
         interpreter is saying something like "Hey! You called Py_Initialize()
         on thread t at 1, you called PyObject_CallMethod() on thread t at 4 and
         now you are calling Py_Finalize() on thread t at 7; what the #@$ are
         you trying to do!?"
    ---> do I need to create a ThreadState object for every OS thread which
         could possibly use the Python Interpreter ? or is it, on per
         concurent thread ?

 4) when executing my message (i.e. logic in threads B or C), what do I
    have to do to lock ? When do I need to lock ? or unlock ?
    Which methods: PyEval_SavedThread(), PyEval_AcquireThread(),
                   PyEval_AcquireLock(), Py_BEGIN_ALLOW_THREADS,.... ?

Any help would be greatly appreciated.

Thanks,

Nicolas Duchastel



More information about the Python-list mailing list