[Numpy-discussion] Why does fancy indexing work like this?

[Sebastian Berg]

On Wed, 2020-07-22 at 16:55 -0600, Aaron Meurer wrote:
> Ah, so I guess I caught this issue right as it got fixed. There are
> no

Yes, on a general note. Advanced indexing grew over time in a maze of
paths, and things like empty arrays were long not too well supported in
many parts of NumPy.  That this went through 

> warnings in 1.19.0, but I can confirm I get the warnings in numpy
> master. 1.19.1 isn't on conda yet, but I tried building it and didn't
> get the warning there. So I guess I need to wait for 0.19.2.

We don't add warnings in minor releases, so 1.19.2 will definitely
never get it.  I did not remember whether it was in there, because it
was merged around the same time 1.19.x was branched.

About your warnings, do you have a nice way to do that?  The mechanism
for warnings does not really give a good way to catch that a warning
was raised and then turn it into an error.  Unless someone contributes
a slick way to do it, I am not sure the complexity pays off.

IIRC, I added the note about raising the warning, because in this
particular case the deprecation warning (turned into an error) happens
to be chained due to implementation details.  (so you do see the
"original" error printed out).

> 
> How long do deprecation cycles like this tend to last (I'm also
> curious when the warnings for things like a[[[0, 1], [0, 1]]] will go
> away)?

Not sure, this is a corner case, and is bugging pandas a bit, so it may
be a bit quicker, but likely still 2 releases?

We are not always good about phasing out deprecations immediately when
it is plausible. The one you mention strikes me as a bigger one though,
so I think we should wait about 2 years.  It is plausible that we are
there already, even for a while.

Cheers,

Sebastian


> 
> Aaron Meurer
> 
> On Wed, Jul 22, 2020 at 4:32 PM Sebastian Berg
> <sebastian at sipsolutions.net> wrote:
> > On Wed, 2020-07-22 at 16:23 -0600, Aaron Meurer wrote:
> > > Why does fancy indexing have this behavior?
> > > 
> > > > > > a = np.empty((0, 1, 2))
> > > > > > b = np.empty((1, 1, 2))
> > > > > > a[np.array([10, 10])]
> > > Traceback (most recent call last):
> > >   File "<stdin>", line 1, in <module>
> > > IndexError: index 10 is out of bounds for axis 0 with size 0
> > > > > > a[:, np.array([10, 10])]
> > > array([], shape=(0, 2, 2), dtype=float64)
> > > > > > a[:, :, np.array([10, 10])]
> > > array([], shape=(0, 1, 2), dtype=float64)
> > > > > > b[np.array([10, 10])]
> > > Traceback (most recent call last):
> > >   File "<stdin>", line 1, in <module>
> > > IndexError: index 10 is out of bounds for axis 0 with size 1
> > > > > > b[:, np.array([10, 10])]
> > > Traceback (most recent call last):
> > >   File "<stdin>", line 1, in <module>
> > > IndexError: index 10 is out of bounds for axis 1 with size 1
> > > > > > b[:, :, np.array([10, 10])]
> > > Traceback (most recent call last):
> > >   File "<stdin>", line 1, in <module>
> > > IndexError: index 10 is out of bounds for axis 2 with size 2
> > > 
> > > As far as I can tell, the behavior is that if an array has a 0
> > > dimension and an integer array index indexes an axis that isn't
> > > 0,
> > > there are no bounds checks. Why does it do this? It seems to be
> > > inconsistent with the behavior of shape () fancy indices (integer
> > > indices). I couldn't find any reference to this behavior in
> > > https://numpy.org/doc/stable/reference/arrays.indexing.html.
> > > 
> > 
> > The reason is because we used to not do this when there are *two*
> > advanced indices:
> > 
> >    arr = np.ones((5, 6))
> >    arr[[], [10, 10]]
> > 
> > giving an empty result.  If you check on master (and maybe on
> > 1.19.x, I
> > am not sure). You should see that all of your examples give a
> > deprecation warning to be turned into an error (except the example
> > I
> > gave above, which can be argued to be correct).
> > 
> > - Sebastian
> > 
> > 
> > > Aaron Meurer
> > > _______________________________________________
> > > NumPy-Discussion mailing list
> > > NumPy-Discussion at python.org
> > > https://mail.python.org/mailman/listinfo/numpy-discussion
> > > 
> > 
> > _______________________________________________
> > NumPy-Discussion mailing list
> > NumPy-Discussion at python.org
> > https://mail.python.org/mailman/listinfo/numpy-discussion
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200722/00001628/attachment-0001.sig>