Python 2.3 problem with extention-embedding in multi-threaded and multi sub-interpreter environment

Atul Kshirsagar atul.kshirsagar at firstlogic.com
Fri Mar 19 12:35:17 EST 2004


I am embedding python in my C++ application. I am using Python *2.3.2* with
a C++ extention DLL in multi-threaded environment. I am using SWIG-1.3.19 to
generate C++ to Python interface.

Now to explain it in details,
1. Python initialization [Py_Initialize()] and finalization [Py_Finalize()]
is done in the *main* thread.
2. For each new thread I create a separate sub-interpreter
[Py_NewInterpreter()]. 
3. Using PyRun_String("import myModule"...) before execution of python
script, extention module 
   is imported.
4. Each thread executes *multiple* python script using PyEval_EvalCode()
using the class objects in
   my extention DLL.
5. Each sub-interpreter is destroyed [Py_EndInterpreter()] at the end of
that particular thread.

I am observing that;
As explained above when multiple threads are running. And as one of these
threads finishes, in other running threads I start getting 
"TypeError: 'NoneType' object is not callable" error on the methods called
on class objects in extention module. 

The same code *works fine* with Python 2.2.2.

I have found these links more or less talking about the same problem
migrating from 2.2 to 2.3.
http://mail.python.org/pipermail/python-dev/2003-September/038237.html
http://mail.python.org/pipermail/python-list/2004-February/206851.html
http://mail.python.org/pipermail/python-list/2004-January/204040.html

I *guess* what is happening is global variables are zapped to "NoneType"
when one thread finishes and other thread trying to access them through the
Python script (step 4.) this error is generated. But it *works* sometimes
when(*guess*) the running thread is at step 3. and by importing the  module
the global variables are re-initialized for "Type" information.

I tried using reload(myModule) to solve the problem but that is generating
big memory leak every time it is called.

Is this a know issue with 2.3 (interpreter?) ? Or is there a change for 2.3
in the way embedding should be done in a multi-threaded and
multi-sub-interpreter environment ?

Anybody for help ?
Thanks,
Atul





More information about the Python-list mailing list