PyGilState_Ensure interrupts python critical sections

[Chris Mellon]

On Nov 15, 2007 4:50 AM,  <billy.omahony at gmail.com> wrote:
> Many thanks for the clarification. Also good idea to focus on lock
> ownership rather that thread activity in your diagram.
>
> To be honest I was not actually experiencing deadlock issues. I had
> just deduced (incorrectly) that I might do so if I started using locks
> in my py code called from c-land. As I was suffering some horrible
> race-condtions that had suddenly appeared in code that had been
> perfectly stable for the past 6 months I didn't want to go down that
> route given my misconceptions; Even if introducing the locks worked
> I'd always feel it was a deadlock waiting to happen.
>

A good reason to avoid the threading metaphor for concurrency entirely ;)

> Misunderstanding arose as for some reason I had decided that python
> code called from C did not allow any other python threads to run until
> control had returned to C and the gil had been released from C-land.
> I.e. once the gil was given to the C thread then python had to wait
> until it was given back.
>

This is essentially true. The part you're missing is that some Python
functions themselves release the GIL and call into C - in this case,
it's blocking on a lock. Some other functions that do this including
blocking on file or (especially) socket IO.

If this didn't happen, it'd be pretty much impossible to mix Python
threads and C threads in any safe manner - the deadlock situation you
envisioned would be a very common reality.

> I have to say the docs could be a little more explicit on the
> mechanisms involved. They tend to be better at stating what individual
> functions do rather that giving overall explanations. Maybe I'll get
> around to submitting a chapter on it ;)
>