[Numpy-discussion] NEP: Dispatch Mechanism for NumPy’s high level API

[Stephan Hoyer]

On Sun, Jun 3, 2018 at 11:12 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> On Sun, Jun 3, 2018 at 2:00 PM, Hameer Abbasi <einstein.edison at gmail.com>
> wrote:
>
>>
>>    - Objects that don’t implement ``__array_function__`` should be
>>    treated as having returned ``np.NotImplementedButCoercible``.
>>       - This has the effect of coercing ``list``, etc.
>>       - At a minimum, to maintain compatibility, if all objects don’t
>>       implement ``__array_function__``, the old behaviour should stay.
>>
>> I think that in the proposed scheme this is effectively what happens.
>

The current proposal is to copy the behavior of __array_ufunc__. So the
non-existence of an __array_function__ attribute is indeed *not* equivalent
to returning NotImplemented: if no arguments implement __array_function__,
then yes they will all be coerced to NumPy arrays.

I do think there is elegance in defining a return value of
np.NotImplementedButCoercible as equivalent to the existence of
__array_function__. This resolves my design question about how coercible
arguments would be coerced with NotImplementedButCoercible: we would fall
back to the current behavior, which in most cases means all arguments are
coerced to NumPy arrays directly. Mixed return values of
NotImplementedButCoercible and NotImplemented would still result in
TypeError, and there would be no second chances for overloads.

This is simple enough that I am inclined to update the NEP to incorporate
the suggestion (thank you!).

My main question is whether we should also update __array_ufunc__ to
support returning NotImplementedButCoercible for consistency. My
inclination is yes: even though it's easy to implement a fallback of
converting all arguments to NumPy arrays for ufuncs, it is hard to do this
correctly from an __array_ufunc__ implementation, because __array_ufunc__
implementations do not know in what order they have been called.

The counter-argument would be that it's not worth adding new features to
__array_ufunc__ if use-cases haven't come up yet. But my guess is that most
users/implementors of __array_ufunc__ are ignorant of these finer details,
and not really worrying about them. Also, the list of binary operators in
Python is short enough that most implementations are OK with supporting
either all or none.

Actually, a return value of NotImplementedButCoercible would probably be
the right answer for some cases in xarray's current __array_ufunc__ method,
when we encounter ufunc methods for which we haven't written an
implementation (e.g., "outer" or "at").
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180603/106b6fb5/attachment-0001.html>