[Python-Dev] thread issues when embedding Python

[Daniel Pocock]

On 18/12/13 16:02, Chris Angelico wrote:
> On Wed, Dec 18, 2013 at 9:26 PM, Daniel Pocock <daniel at pocock.com.au> wrote:
>> b) when each worker thread starts, call
>> PyThreadState_New(mInterpreterState)  and save the result in a thread
>> local mPyThreadState
>>
>> c) use the mPyThreadState with PyEval_RestoreThread and
>> PyEval_SaveThread before and after calling Python methods
>>
>> Is this a bug in PyGILState_Ensure or is it a deficiency in the
>> documentation?
> It doesn't surprise me that you would need to do step b - I do seem to
> recall the need to call that for each new thread. Not so sure about c.
> Once you fixed the unrelated bug, do you still need that step? (Been a
> while since I last embedded Python though, and I might well be wrong.)

Yes, I definitely needed to use this PyThreadState_New call even after
my unrelated bug fix

Should it be added to the documentation?

I created a C++ wrapper around this logic, it is here:

https://github.com/resiprocate/resiprocate/blob/master/repro/plugins/pyroute/PyThreadSupport.hxx

and the use case is something like:

      // in constructor:

    PyExternalUser* mPyUser = new PyExternalUser(mInterpreterState);


and each time Python calls are made, just do:

{

        PyExternalUser::Use use(*mPyUser);
        // now call Python code
}

When the PyExternalUser::Use instance is created it does PyEval_RestoreThread()

When the PyExternalUser::Use instance goes out of scope it is destroyed and PyEval_SaveThread() is called