[Numpy-discussion] Should unique types of all arguments be passed on in __array_function__?

[Stephan Hoyer]

On Mon, Nov 5, 2018 at 9:00 AM Marten van Kerkwijk <
m.h.vankerkwijk at gmail.com> wrote:

> Hi Stephan,
>
> I fear my example about thinking about `ndarray.__array_function__`
> distracted from the gist of my question, which was whether for
> `__array_function__` implementations *generally* it wouldn't be handier to
> have all unique types rather than just those that override
> `__array_function__`. It would seem that for any other implementation than
> for numpy itself, the presence of __array_function__ is indeed almost
> irrelevant. As a somewhat random example, why would it, e.g., for DASK be
> useful to know that another argument is a Quantity, but not that it is a
> file handle? (Presumably, it cannot handle either...)
>

In practice, it is of course easy to simply ignore arguments that don’t
define __array_function__. But I do think the distinction is important for
more than merely ndarray: the value of the types argument tells you the set
of types that might have a conflicting implementation.

For example, Dask might be happy to handle any non-arrays as scalars (like
NumPy), e.g., it should be fine to make a dask array consisting of a
decimal object. Since decimal doesn’t define __array_function__, there’s no
need to do anything special to handle it inside
dask.array.Array.__array_function__. If decimal appeared in types, then
dask would have to be careful to let arbitrary types that don’t define
__array_function__ pass through.

In contrast, dask definitely wants to know if another type defines
__array_function__, because they might have a conflicting implementation.
This is the main reason why we have the types argument in the first place —
to make these checks easy. In my experience, it is super common for Python
arithmetic methods to be implemented improperly, i.e., never returning
NotImplemented. This will hopefully be less common with __array_function__.

More broadly, it is only necessary to reject an argument type at the
__array_function__ level if it defines __array_function__ itself, because
that’s the only case where it would make a difference to return
NotImplemented rather than trying (and failing) to call the overriden
function implementation.



>
> All the best,
>
> Marten
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20181109/de357598/attachment.html>