[Numpy-discussion] defining a NumPy API standard?

[Ralf Gommers]

On Sat, Jun 1, 2019 at 10:32 PM Nathaniel Smith <njs at pobox.com> wrote:

> On Sat, Jun 1, 2019, 09:13 Ralf Gommers <ralf.gommers at gmail.com> wrote:
>
>>
>>
>> On Sat, Jun 1, 2019 at 11:32 AM Nathaniel Smith <njs at pobox.com> wrote:
>>
>>> It's possible I'm not getting what you're thinking, but from what you
>>> describe in your email I think it's a bad idea.
>>>
>>
>> Hi Nathaniel, I think you are indeed not getting what I meant and are
>> just responding to the word "standard".
>>
>
> Well, that's the word you chose :-)
>

It's just one word out of 100 line email. I'm happy to retract it. Please
pretend it wasn't there and re-read the rest. Replace it with the list of
functions that I propose in my previous email.


> I think it's very possible that what you're thinking is a good idea, but
> it's actually something else, like better high-level documentation, or a
> NEP documenting things we wish we did differently but are stuck with, or a
> generic duck array test suite to improve compatibility and make it easier
> to bootstrap new libraries, etc.
>
> The word "standard" is tricky:
>
> - it has a pretty precise technical meaning that is different from all of
> those things, so if those are what you want then it's a bad word to use.
>
> - it's a somewhat arcane niche of engineering practice that a lot of
> people don't have direct experience with, so there are a ton of people with
> vague and magical ideas about how standards work, and if you use the word
> then they'll start expecting all kinds of things. (See the response up
> thread where someone thinks that you just proposed to make a bunch of
> incompatible changes to numpy.) This makes it difficult to have a
> productive discussion, because everyone is misinterpreting each other.
>
> I bet if we can articulate more precisely what exactly you're hoping to
> accomplish,
>

Please see my email of 1 hour ago.

>
>
>> I'll give a concrete example. Here is the xtensor to numpy comparison:
>> https://xtensor.readthedocs.io/en/latest/numpy.html. The xtensor authors
>> clearly have made sane choices, but they did have to spend a lot of effort
>> making those choices - what to include and what not.
>>
>> Now, the XND team is just starting to build out their Python API. Hameer
>> is building out unumpy. There's all the other arrays libraries I mentioned.
>> We can say "sort it out yourself, make your own choices", or we can provide
>> some guidance. So far the authors of those libaries I have asked say they
>> would appreciate the guidance.
>>
>
> That sounds great. Maybe you want... a mailing list or a forum for array
> library implementors to compare notes?
>

No.

("So we ran into this unexpected problem implementing einsum, how did you
> handle it? And btw numpy devs, why is it like that in the first place?")
>

can be done on this list.

Maybe you want someone to write up a review of existing APIs like xtensor,
> dask, xarray, sparse, ... to see where they deviated from numpy and if
> there are any commonalities?
>

That will be useful in verifying that the list of functions for "core of
NumPy" I proposed is sensible. We're not going to make up things out of
thin air.


> Or someone could do an analysis of existing code and publish tables of how
> often different features are used, so array implementors can make better
> choices about what to implement first?
>

That's done:)
NumPy table:
https://github.com/Quansight-Labs/python-api-inspect/blob/master/data/csv/numpy-summary.csv
Blog post with explanation:
https://labs.quansight.org/blog/2019/05/python-package-function-usage/

And yes, it's another useful data point in verifying our choices.

Or maybe just encouraging Hameer to be really proactive about sharing
> drafts and gathering feedback here?
>

No. (well, it's always good to be proactive, but besides the point here)

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190601/27be46db/attachment.html>