[Cython] Fwd: Re: [cython-users] checking for "None" in nogil function

[Stefan Behnel]

Dag Sverre Seljebotn, 07.05.2012 12:40:
> moving to dev list

Makes sense.

> On 05/07/2012 11:17 AM, Stefan Behnel wrote:
>> Dag Sverre Seljebotn, 07.05.2012 10:44:
>>> On 05/07/2012 07:48 AM, Stefan Behnel wrote:
>>>> I wonder why a memory view should be allowed to be None in the first
>>>> place.
>>>> Buffer arguments aren't (because they get unpacked on entry), so why
>>>> should memory views?
>>>
>>> ? At least when I implemented it, buffers get unpacked but the case of a
>>> None buffer is treated specially, and you're fully allowed (and segfault if
>>> you [] it).
>>
>> Hmm, ok, maybe I just got confused by the code then.
>>
>> I think the docs should state that buffer arguments are best used together
>> with the "not None" declaration then.

... which made me realise that that wasn't even supported. I can't believe
no-one ever reported that as a bug...

https://github.com/cython/cython/commit/f2de49fd0ac82a02a070b931bf4d2dab47135d0b

It's still not supported for memory views.

BTW, is there a reason why we shouldn't allow a "not None" declaration for
cdef functions? Obviously, the caller would have to do the check in that
case. Hmm, maybe it's not that important, because None checks are best done
at entry points from user code, which usually means Python code. It seems
like "not None" is not supported on cpdef functions, though.


> I use them with "=None" default values all the time... then do a
> None-check manually.

Interesting. Could you given an example? What's the advantage over letting
Cython raise an error for you? And, since you are using it as a default
argument, why would someone want to call your code entirely without a
buffer argument?


> It's really no different from cdef classes.

I find it at least a bit more surprising because a buffer unpacking
argument is a rather strong hint that you expect something that supports
this protocol. The fact that you type your function argument with it hints
at the intention to properly unpack it on entry. I'm sure there are lots of
users who were or will be surprised when they realise that that doesn't
exclude None values.


>> And I remember that we wanted to change the default settings for extension
>> type arguments from "or None" to "not None" years ago but never actually
>> did it.
> 
> I remember that there was such a debate, but I certainly don't remember
> that this was the conclusion :-)

Maybe not, yes.


> I didn't agree with that view then and
> I don't now. I don't remember what Robert's view was...
> 
> As far as I can remember (which might be biased towards my personal
> view), the conclusion was that we left the current semantics in place,
> relying on better control flow analysis to make None-checks cheaper, and
> when those are cheap enough, make the nonecheck directive default to
> True

At least for buffer arguments, it silently corrupts data or segfaults in
the current state of affairs, as you pointed out. Not exactly ideal.

That's another reason why I see a difference between the behaviour of
extension types and that of buffer arguments. Buffer indexing is also way
more performance critical than the average method call or attribute access
on a cdef class.


> (Java is sort of prior art that this can indeed be done?).

Java was designed to have a JIT compiler underneath which handles external
parameters, and its compilers are way smarter than Cython. I agree that
there is still a lot we can do based on better static analysis, but there
will always be limits.

Stefan