[Cython] Calling gil-requiring function not allowed without gil

[Robert Bradshaw]

On Wed, Aug 17, 2011 at 11:46 AM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 08/17/2011 08:19 PM, Robert Bradshaw wrote:
>>
>> That's a nice idea. I have to admit that all these special gil
>> declarations are a bit messy. I'd also rather introduce clear
>> decorators, e.g.
>>
>> @cython.requires_gil  # expects gil
>> cdef a(): ...
>>
>> @cython.requires.gil(False) # nogil
>> cdef b(): ...
>>
>> @cython.aquires_gil  # with gil
>> cdef c(): ...
>>
>> (Actually, now that we have the "with gil" statement, it could be
>> worth considering simply noticing the pattern of the entire function
>> body in a with gil block/as the first statement and acquiring the GIL
>> before argument parsing.)
>>
>> Note that we need to declare functions as requiring the GIL to allow
>> for declaring cpython.pxd if extern functions are implicitly nogil.
>
> I agree, it's messy in the current situation, simplifying would be good.
>
> Assuming we can't acquire the GIL in every single function just to be sure,
> I have a hunch that the "acquires_gil" aspect of a function is just declared
> in the wrong place. I mean, the same function might be passed as a callback
> to C both while holding the GIL and while not holding the GIL -- it would be
> nice to automatically wrap it in a GIL-acquiring wrapper only when needed.
>
> So to me it makes more sense to have acquires_gil be part of function
> pointer types, or of the C call where the pointer is passed, or similar.
> Sort of like an FFI. Can't think of a practical scheme that's more
> user-friendly than the current way though...

I was thinking the opposite, "aquires_gil" should be completely
transparent to the caller (assuming it's cheap enough to
check-or-acquire, but that's an optimization not API issue). On the
other hand requires/does not require does need to be visible to the
caller though, which argues for it being part of the signature.

Regarding inference, are you thinking the semantics being that we
(re)-acquire the GIL for every individual operation that needs it
(including python operations or entire with gil blocks), with the
obvious optimization that we may choose to not release it between
(nearly) consecutive blocks of code that all need the GIL? That would
be truer to Python semantics, but would require risky guesswork at
compile time or some kind of periodic runtime checks. (It would also
be a strongly backwards incompatible move, so as mentioned you'd need
some kind of a explicit marker and deprecation period.)

- Robert