[Numpy-discussion] NEP-18 comment

[Sebastian Berg]

On Thu, 2019-03-07 at 18:24 +0000, Frederic Bastien wrote:
> I see speed changes vs behavior changes as different category of
> changes in my mind.
> 
> I understand that now importing library can slow down NumPy for small
> arrays.
> But I have the impression you tell this can also give behavior
> change.
> 
> I do not understand why this could happen. A pure numpy script, that
> you just import dask or other library without using them, would just
> cause a slowdown. Not a behavior change.
> 
> Behavior change's start to happen only when you start to use the new
> library.


Yes, but if a user can switch, downstream projects might change their
behaviour (my thought makes that less likely, but it is still a risk).

The best solution would be to just make things fast enough that we do
not have to worry about it. And I would not rule out that we are
already there.
Things seem a bit hard to time with all the C/Python boundaries
involved, but it seems to me that the actual time spend on the C-side
is reasonably low. The biggest chunk is probably that we have 4
function calls instead of 1:

  1. Public api call
  2. dispatch call (get arguments relevant for dispatching)
  3. Call C-side function implementing the logic
  4. Call the actual function

Improving that much seems like it might get ugly pretty fast. But I am
wondering if dispatching from within C for functions that are currently
defined in C already may be moderately easy (and also the ones with the
largest pay-off).

Best,

Sebastian



> 
> Did I miss something?
> 
> Frédéric
> 
> -----Original Message-----
> From: NumPy-Discussion <
> numpy-discussion-bounces+fbastien=nvidia.com at python.org> On Behalf Of
> Stefan van der Walt
> Sent: Thursday, March 7, 2019 12:15 PM
> To: Discussion of Numerical Python <numpy-discussion at python.org>
> Cc: Matthew Rocklin <mrocklin at gmail.com>
> Subject: Re: [Numpy-discussion] NEP-18 comment
> 
> Hi Sebastian, Frederic,
> 
> On Thu, 07 Mar 2019 14:23:10 +0000, Frederic Bastien wrote:
> > I like your idea Sebastian. This way it is enabled only when needed
> > and it is invisible to the user at the same time.
> > 
> > Stefan, does it solve well enough the potential problem you raised?
> 
> I don't think so.  This means that NumPy suddenly behaves differently
> when dask is imported, which again causes the problem mentioned
> earlier:
> that identical NumPy code could behave differently depending on
> library versions, imports, and the environment.
> 
> That said, I think this is a better solution than an environment
> variable.
> 
> Anyway, my opinion is just one of many: I'd like to hear what the
> other developers think.
> 
> Best regards,
> Stéfan
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> -------------------------------------------------------------------
> ----------------
> This email message is for the sole use of the intended recipient(s)
> and may contain
> confidential information.  Any unauthorized review, use, disclosure
> or distribution
> is prohibited.  If you are not the intended recipient, please contact
> the sender by
> reply email and destroy all copies of the original message.
> -------------------------------------------------------------------
> ----------------
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20190307/5583b042/attachment.sig>