[C++-sig] PyGILState_Release with multithread

[Matthew Scouten (TT)]

First of all, the PyGILState_* functions are acquiring and releasing the "Global Interpreter _Lock_" Your _pyMutex is redundant. 

Here are the rules that govern the GIL:

1)      Only one thread at a time can hold the GIL

2)      Any thread that is touching python data, python code, or any part of the 'terp must hold the GIL

 

The python part of your program WILL be single threaded. There is nothing you can do about that. 

 

From: cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org [mailto:cplusplus-sig-bounces+matthew.scouten=tradingtechnologies.com at python.org] On Behalf Of Pascal Briet
Sent: Tuesday, February 02, 2010 5:46 AM
To: cplusplus-sig at python.org
Subject: [C++-sig] PyGILState_Release with multithread

 

Hello,

 

After a few days of headache, I think the best way is to share this problematic...

 

I have :

- a main Python script

- a C module imported and called from Python  (thanks to boost ::python)

- A dozen of threads created by a C function  (boost ::thread)

- These threads, in a C main loop, call regularly a Python function.

 

 

Here is the critical part of the C code, with the Python GIL management :

 

boost::mutex      _pyMutex;

PyGILState_STATE  _gstate;

 

_pyMutex.lock();

_gstate = PyGILState_Ensure();

_pyMutex.unlock();

 

// Interprets Python bytecodes with boost::python

 

_pyMutex.lock();

PyGILState_Release(_gstate);

_pyMutex.unlock();

 

 

This part is multi-threaded, and it seems that the PyGILState_* functions do not like it.

I have the following error with PyGILState_Release : « This thread state must be current when releasing »

 

I think that when 2 threads enter PyGILState_Ensure, the second one is considered as the current one. When the first one end...  it is not current. A quick look to the code of the Python API seems to confirm it.

 

 

So, the only solution I found is to lock the whole call to Python :

 

boost::mutex      _pyMutex; 

PyGILState_STATE  _gstate;

 

_pyMutex.lock();

_gstate = PyGILState_Ensure();

 

// Interprets Python bytecodes with boost::python

 

PyGILState_Release(_gstate);

_pyMutex.unlock();

 

 

Everything is ok with this solution, except that...  only one thread at a time can call a Python method.

When 2 threads ask for a Python call, if the first one is a blocking operation, the second one will wait...  it is so bad !

 

Do you have any solution ?

 

Nb : I use Python 2.5.2

 

Thanks for your help.

 

 

_____

 

Briet Pascal

Ingénieur d'études

 

+33 (0)1 47 42 10 55

 

OptimProcess

32 rue Tronchet - 75009 Paris 
www.optimprocess.com <http://www.optimprocess.com/> 

 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100202/caf3ed57/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: image/jpeg
Size: 2121 bytes
Desc: image001.jpg
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100202/caf3ed57/attachment-0001.jpeg>