[ python-Bugs-921077 ] embedding in multi-threaded & multi sub-interpreter environ

[SourceForge.net]

Bugs item #921077, was opened at 2004-03-22 16:39
Message generated for change (Comment added) made by loewis
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=921077&group_id=5470

Category: None
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: Atul (atulkshirsagar)
Assigned to: Nobody/Anonymous (nobody)
>Summary: embedding in multi-threaded & multi sub-interpreter environ

Initial Comment:
I am embedding python in my C++ application. I am 
using Python *2.3.2* with a C++ extention DLL in multi-
threaded environment. I am using SWIG-1.3.19 to 
generate C++ to Python interface.

Now to explain it in details,
1. Python initialization [Py_Initialize()] and finalization 
[Py_Finalize()] is done in the *main* thread.
2. For each new thread I create a separate sub-
interpreter [Py_NewInterpreter()]. 
3. Using PyRun_String("import myModule"...) before 
execution of python script, extention module is imported.
4. Each thread executes *multiple* python script using 
PyEval_EvalCode() using the class objects in my 
extention DLL.
5. Each sub-interpreter is destroyed [Py_EndInterpreter
()] at the end of that particular thread.

I am observing that;
As explained above when multiple threads are running. 
And as one of these threads finishes, in other running 
threads I start getting 
"TypeError: 'NoneType' object is not callable" error on 
the methods called on class objects in extention module. 

The same code *works fine* with Python 2.2.2.

I have found these links more or less talking about the 
same problem
migrating from 2.2 to 2.3.
http://mail.python.org/pipermail/python-dev/2003-
September/038237.html
http://mail.python.org/pipermail/python-list/2004-
February/206851.html
http://mail.python.org/pipermail/python-list/2004-
January/204040.html

I *guess* what is happening is global variables are 
zapped to "NoneType"
when one thread finishes and other thread trying to 
access them through the
Python script (step 4.) this error is generated. But it 
*works* sometimes
when(*guess*) the running thread is at step 3. and by 
importing the  module
the global variables are re-initialized for "Type" 
information.

I tried using reload(myModule) to solve the problem but 
that is generating
big memory leak every time it is called.

Is this a know issue with 2.3 (interpreter?) ? Or is there 
a change for 2.3 in the way embedding should be done 
in a multi-threaded and multi-sub-interpreter 
environment ?


----------------------------------------------------------------------

>Comment By: Martin v. Löwis (loewis)
Date: 2004-03-30 00:28

Message:
Logged In: YES 
user_id=21627

Can you provide an example that demonstrates the supposed bug?

----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=921077&group_id=5470