Python callbacks & PyGILState_Release()

[David E. Konerding DSD staff]

In article <mailman.2345.1114265296.1799.python-list at python.org>, Randall Hopper wrote:
> Thomas Heller:
> |>      Python -> C++ -> Python Callback
> |>
> |> (example attached) an exception raised in the callback doesn't make it back
> |> across C++ to Python.
> ...
> |> void callback_wrapper( void *user_data )
> |> {
> |>   // Acquire interpreter lock
> |>   PyGILState_STATE gstate = PyGILState_Ensure();
> |>   ...
> |>   // Call Python
> |>   pyresult = PyEval_CallObject( pyfunc, pyargs );
> |>   ...
> |
> |    if (pyresult == NULL)
> |        PyErr_Print();
> |
> |>   // Free interpreter lock
> |>   PyGILState_Release(gstate);
> |> }
> |
> |PyErr_Print() will do the 'right' thing?s.
> 
> Thanks for the reply.  However, this won't:
> 
>    a) Stop the main Python script, and
>    b) Print the full stack trace (including Python and C++ SWIG wrapper)
> 
> Is there a clean way to save the full exception state in the callback
> before the PyGILState_Release(), and restore it when we return across the
> C++ wrapper?
> 
> If I knew what the proper "save" and "restore" exception state code bits
> were, I could easily implement this with exception typemaps in SWIG.
> 
> Thanks,
> 
> Randall
> 
> P.S. Perhaps PyGILState_Release should take an argument instructing it to
> exclude exception state when resetting the interpreter state back to its
> original state.

Randall:

It's not the job of the PyGILState_* functions to manage exception details for you.

I always solved this problem a different way, by saving the exception in an instance variable
within the Python callback, and using a condition variable so that the main thread could abort on a callback's
failure.  Notably, our callsbacks are always invoked from a C++ thread that the main Python interepreter didn't
create, and our main Python intepreter is blocked on a mutex most of the time.

I saved the exception state by retrieveing it from sys.exc_info(), which contains all the traceback object data as Python variable.
I think you can get the same info from your C++ callback wrapper, and use PyErr_Fetch and PyErr_Restore to save and restore the
exception sate.

Dave