[Numpy-discussion] defining a NumPy API standard?

[Nathaniel Smith]

On Sat, Jun 1, 2019, 09:13 Ralf Gommers <ralf.gommers at gmail.com> wrote:

>
>
> On Sat, Jun 1, 2019 at 11:32 AM Nathaniel Smith <njs at pobox.com> wrote:
>
>> It's possible I'm not getting what you're thinking, but from what you
>> describe in your email I think it's a bad idea.
>>
>
> Hi Nathaniel, I think you are indeed not getting what I meant and are just
> responding to the word "standard".
>

Well, that's the word you chose :-)

I think it's very possible that what you're thinking is a good idea, but
it's actually something else, like better high-level documentation, or a
NEP documenting things we wish we did differently but are stuck with, or a
generic duck array test suite to improve compatibility and make it easier
to bootstrap new libraries, etc.

The word "standard" is tricky:

- it has a pretty precise technical meaning that is different from all of
those things, so if those are what you want then it's a bad word to use.

- it's a somewhat arcane niche of engineering practice that a lot of people
don't have direct experience with, so there are a ton of people with vague
and magical ideas about how standards work, and if you use the word then
they'll start expecting all kinds of things. (See the response up thread
where someone thinks that you just proposed to make a bunch of incompatible
changes to numpy.) This makes it difficult to have a productive discussion,
because everyone is misinterpreting each other.

I bet if we can articulate more precisely what exactly you're hoping to
accomplish, then we'll also be able to figure out specific concrete actions
that will help, and they won't involve the word "standard". For example:


> I'll give a concrete example. Here is the xtensor to numpy comparison:
> https://xtensor.readthedocs.io/en/latest/numpy.html. The xtensor authors
> clearly have made sane choices, but they did have to spend a lot of effort
> making those choices - what to include and what not.
>
> Now, the XND team is just starting to build out their Python API. Hameer
> is building out unumpy. There's all the other arrays libraries I mentioned.
> We can say "sort it out yourself, make your own choices", or we can provide
> some guidance. So far the authors of those libaries I have asked say they
> would appreciate the guidance.
>

That sounds great. Maybe you want... a mailing list or a forum for array
library implementors to compare notes? ("So we ran into this unexpected
problem implementing einsum, how did you handle it? And btw numpy devs, why
is it like that in the first place?") Maybe you want someone to write up a
review of existing APIs like xtensor, dask, xarray, sparse, ... to see
where they deviated from numpy and if there are any commonalities? Or
someone could do an analysis of existing code and publish tables of how
often different features are used, so array implementors can make better
choices about what to implement first? Or maybe just encouraging Hameer to
be really proactive about sharing drafts and gathering feedback here?

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190601/3eee2bb1/attachment-0001.html>