Multithreaded C API Python questions

[Svein Seldal]

robert wrote:

> Forget all the low level PyGIL... functions.'

Quite the contrary, I would say!

 From the Py C API doc, I interpret the PyGIL... functions to be higher 
leveled than eg. PyEval_SaveThread(). I've checked into the source of 
python to find out what's really going on, and I've learnt a couple of 
interesting facts. The PyGIL... are helper functions build over the 
PyEval... functions, not the other way around.

A real beauty about the PyGILState_Ensure() is the fact that if it's 
called from a thread that is (yet) unknown to python, it will actually 
create the thread state object for you (with PyThreadState_New) and 
acquire the lock. Hence, I dont have to worry about the process of 
swapping thread states with PyEval_xxxThread() functions. I ONLY need to 
use PyGILState_Ensure() prior to any py-ops.

 From my previous post, I asked about the difference between 
PyEval_ReleaseLock() and PyEval_ReleaseThread() and I've found one major 
difference. Both release the GIL as the docs states, but the 
PyEval_ReleaseThread() saves the current thread state as well. When I 
used PyEval_ReleaseLock() this didnt happen, preventing proper saving of 
the current thread, causing py crash when control were handed back to 
this thread.

By using:

	PyEval_InitThreads();
	py_ops();
	PyThreadState *pts = PyGILState_GetThisThreadState();
	PyEval_ReleaseThread(pts);

And for each py-op later on (from arbitrary thread):

	PyGILState_STATE gstate;
	gstate = PyGILState_Ensure();
	py_ops();
	PyGILState_Release(gstate);

Then you're home free. Everything related to threading is handled by 
Py/API itself.

> Probably you just do a PyRun_xxxx in main thread and then everything 
> else in Python, and expose C-parts for the thread-loop to Python as 
> function (in other_py_inits) - where in the c-function you probably have 
> the usual Py_BEGIN_ALLOW_THREADS/Py_END_ALLOW_THREADS  bracket during 
> time consuming/system/reenter-endangered message stuff.

The app i'm building is a plugin to a server system, and I cant control 
nor remove any threads that the server uses. I am given one main thread 
to run python forever, and messages that are to be delivered into python 
are called from another.

I could do it like you propose: The data coming from my server will 
arrive into a c-function called by a server thread. The c-part of the 
thread-loop would then be run as another thread (started from  python). 
Its fully feasible, yet the challenge is to make proper data sync-ing 
between these two threads.

Well, I think I've achieved what I wanted. Python seems apparently 
stable, and reading from the python sources, I cant see any immediate 
reasons why it shouldn't.

Thanks for letting me closer to a working solution!


Regads,
Svein