c bindings with non-python thread callback while python exits

[Paul Grinberg]

I have a C++ library (luckily with source code, so I know what's going on) to which I wrote c bindings. The library internally starts an event pthread which generates callbacks back into python. The c binding for processing these callbacks look like this (simplified for convenience):

PyGILState_STATE gstate;
gstate = PyGILState_Ensure();
PyObject_CallObject(cb, arglist);
PyGILState_Release(gstate);

This works perfectly while the python program using this c binding is running. However, when the program terminates normally and there happens to be one last callback underway, the c bindings hang. When viewed with GDB, I can see that the callback thread callstack looks like this:

futex_abstimed_wait_cancelable()
_pthread_cond_wait_common()
__pthread_cond_timedwait()
PyCOND_TIMEDWAIT()
take_gil()
PyEval_RestoreThread()
PyGILState_Ensure()
c_bindings_callback()
cpp_library_callback()

and the main thread callstack, which is where the python interpreter was running, looks like this

__pthread_clockjoin_ex ()
std::thread::join()
cpp_library_static_singleton_class_destructor ()
__run_exit_handlers ()
__GI_exit ()
__libc_start_main ()
_start ()

in other words, the program hangs because the main thread is waiting for the event pthread to join, but that thread is stuck in a callback waiting for the GIL.

What is the right way to prevent this problem from happening?
Thank you in advance,
Paul.

P.S. I am running on Linux: ubuntu 18.04 with python 3.6.9, also reproduced with python 3.7.5, as well as ubuntu 20.04 with python 3.8.5