[Numpy-discussion] NEP: array API standard adoption (NEP 47)

[Ralf Gommers]

On Wed, Mar 10, 2021 at 6:41 PM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> Top Posting, to discuss post specific questions about NEP 47 and
> partially the start on implementing it in:
>
>     https://github.com/numpy/numpy/pull/18585
>
> There are probably many more that will crop up. But for me, each of
> these is a pretty major difficulty without a clear answer as of now.
>

All great questions, that Sebastian. Let me reply to the questions that
Aaron didn't reply to inline below.


> 1. I still need clarity how a library is supposed to use this namespace
> when the user passes in a NumPy array (mentioned before).  The user
> must get back a NumPy array after all.  Maybe that is just a decorator,
> but it seems important.
>

I agree that it will be a common pattern that libraries will accept all
standard-compliant array types plus numpy.ndarray. And the output array
type should match the input type. In Aaron's implementation the new array
object has a numpy.ndarray as private attribute, so that's the instance
that should be returned. A decorator seems like a sensible way to handle
that. Or a simple utility function, something like `return
correct_arraytype(out)`.

Either way, that pattern should be added to NEP 47. I don't see a
fundamental problem here, we just need to find the nicest UX for it.

3. For all other function, the same problem applies. You don't actually
> have anything to fix NumPy promotion rules.  You could bake your own
> cake here for numeric types, but I am not sure, you might also need NEP
> 43 in all its promotion power to pull it off.
>

This is probably the single most difficult question implementation-wise.
Note that there are only numerical dtypes (plus boolean), so dealing with
string, datetime, object or third-party dtypes is a non-issue.

4. The PR makes no attempt at handling binary operators in any way
> aside from greedily coercing the other operand.
>

Agreed. This is the same point as (3) I think - how to handle dtype
promotion is the main open question.


> 5. What happens with a mix of array-likes or even array subclasses like
> `astropy.quantity`?
>

Array-likes (e.g. list) should raise an exception, the NEP clearly says "do
not accept array_like dtypes". This is what every other array/tensor
library already does.

Array subclasses should work as expected, assuming they're valid subclasses
and not things like np.matrix. Using Mypy will help avoid writing more
subclasses that break the Liskov substitution principle. More comments in
https://numpy.org/neps/nep-0047-array-api-standard.html#the-asarray-asanyarray-pattern

Mixing two different types of arrays into a single function call should
raise an exception. A design goal is: enable writing functions
`somefunc(x1, x2)` that work for any type of array where `x1, x2` come from
the same library = so they're either the same type, or two types for which
the library itself knows how to mix them. If x1 and x2 are from different
libraries, this will raise an exception.

To be clear, it is not intended that `np.array_api.somefunc(x_cupy)` works
- this will raise an exception.

Cheers,
Ralf



>
> I don't think we have to figure out everything up-front, but I do think
> there are a few very fundamental questions still open, at least for me
> personally.
>
> Cheers,
>
> Sebastian
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/numpy-discussion/attachments/20210311/e4ad1ec9/attachment.html>