reentrant callbacks from C into Python

[gb at cs.unc.edu]

I'm developing a wrapper for the Microsoft ActiveAccessibility, and
SetWinEventHook API's. Most things are working fine but occasionally I
get a reentrant callback on a WinEvent. They advise that this is
possible and say users should guard against it but don't say how.

Here's how it happens:

On SetWinEventHook, I squirrel away a pointer to the Python callable
(call it "phook") that is provided for the callback. The C function
SetWinEventHook is called with a C callback function "chook".

When chook is called from somewhere outside of python, it first
acquires the GIL, the calls phook (back into python), when the
phook returns, chook releases the GIL.

Problem is that some of the functions in the API can cause the
callback to be reentered. So, I'm processing a callback and one of the
calls in the Python code causes the C callback to be reentered.

Now when I try to acquire the GIL, deadlock ensues. 

I've made a hack workaround with a static variable keeping track of
the number of times I've locally acquired the GIL. If I already have
it I don't try to get it again. That appears to work but I wonder if
it would work in the presence of threads.

I thought about releasing the GIL around the API functions that I am
wrapping but they warn the reentry might happen in lots of functions
outside the ActiveAccessibility and WinEvents APIs. So it is hard to
be sure that the GIL won't already be held when I enter the C
callback.

There *must* be a better way. Is there a standard way to deal with a
problem like this?

Thanks
Gary Bishop
Associate Professor
Department of Computer Science
University of North Carolina at Chapel Hill