[Cython] 'with gil:' statement

[Dag Sverre Seljebotn]

On 03/16/2011 12:54 PM, mark florisson wrote:
> On 16 March 2011 11:58, Dag Sverre Seljebotn<d.s.seljebotn at astro.uio.no>  wrote:
>> On 03/16/2011 11:28 AM, mark florisson wrote:
>>
>> I implemented the 'with gil:' statement, and have added error checking
>> for nested 'with gil' or 'with nogil' statements. For instance, with
>> the patch applied Cython wil issue an error when you have e.g.
>>
>> with nogil:
>>      with nogil:
>>          ...
>>
>> (or the same thing for 'with gil:'). This because nested 'nogil' or
>> 'gil' blocks, without intermediate GIL-acquiring/releasing, will abort
>> the Python interpreter. However, if people are acquiring the GIL
>> manually in nogil blocks and then want to release the GIL with a
>> 'nogil' block, it will incorrectly issue an error. I do think this
>> kind of code is uncommon, but perhaps we want to issue a warning
>> instead?
>>
>> I think we should make nested nogil-s noops, i.e.
>>
>> with nogil:
>>      with nogil: # =>  if True:
>>
>> This is because one may want to change "with nogil" to "with gil" for
>> debugging purposes (allow printing debug information).
> Interesting, that does sound convenient, but I'm not if mere
> convenience should move us to simply ignore what is technically most
> likely incorrect code (unless there is intermediate manual locking).

I'm not sure if I understand what you mean here. How does simply 
ignoring redundant "with [no]gil" statements cause incorrect code? Or do 
you mean this is a

I'm just trying to minimize the "language getting in your way" factor. 
It is pretty useless to write

if x:
     if x:
         ...

as well, but Python does allow it.

Warnings on nested "with nogil" is more the role of a "cylint" in my 
opinion.


> In any case, I wouldn't really be against that. If you simply want to
> allow this for debugging, we could also allow print statements in
> nogil sections, by either rewriting it using 'with gil:', or by
> inserting a simple printf (in which case you probably want to place a
> few restrictions).

It's not only print statements. I.e., if I think something is wrong with 
an array, I'll stick in code like

print np.std(x), np.mean(x), np.any(np.isnan(x))

or something more complex that may require temporaries. Or even plot the 
vector:

plt.plot(x)
plt.show() # blocks until I close plot window

Or, launch a debugger:

if np.any(np.isnan(x)):
      import pdb; pdb.set_trace()

so I guess I should stop saying this is only about printing. In general, 
it's nice to use Python during debugging. I find myself replacing "with 
nogil" with "if True" in such situations, to avoid reindenting.

I guess I can soon start using "with gil" around the debug code though. 
Again, the restriction on nested nogil/gil is not a big problem, just an 
instance of "the language getting in your way".

Dag