[Numpy-discussion] Inconsistent behavior for ufuncs in numpy v1.10.X

[Sebastian Berg]

On Di, 2016-01-26 at 17:27 +0000, Solbrig,Jeremy wrote:
> Hello Chuck,
> 
> I receive the same result with 1.10.4.  I agree that it looks like
> __array_prepare__, __array_finalize__, and __array_wrap__ have not
> been changed.  I’m starting to dig into the source again, but
> focusing on the _MaskedBinaryOperation class to try to understand
> what is going on there.
> 

Well, there was definitely a change in that code that will cause this,
i.e. the code block:

        if isinstance(b, MaskedArray):
            if isinstance(a, MaskedArray):
                result._update_from(a)
            else:
                result._update_from(b)
        elif isinstance(a, MaskedArray):
            result._update_from(a)

was changed to something like:

        masked_result._update_from(result)

My guess is almost that this fixed some other bug (but you should
probably check git blame to see why it was put in). It might be that
_optinfo should be handled more specifically. It seems like a very
weird feature to me though when the info is always copied from the left
operand...

Is _optinfo even *documented* to exist? Because frankly, unless it is
used far more, and the fact that it is hidden away with an underscore,
I am not sure we should prioritize that it would keep working,
considering that I am unsure that it ever did something very elegantly.

- Sebastian



> Jeremy
> 
> From: NumPy-Discussion <numpy-discussion-bounces at scipy.org> on behalf
> of Charles R Harris <charlesr.harris at gmail.com>
> Reply-To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Date: Tuesday, January 26, 2016 at 10:17 AM
> To: Discussion of Numerical Python <numpy-discussion at scipy.org>
> Subject: Re: [Numpy-discussion] Inconsistent behavior for ufuncs in
> numpy v1.10.X
> 
> 
> 
> On Tue, Jan 26, 2016 at 10:11 AM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
> > 
> > 
> > On Mon, Jan 25, 2016 at 10:43 PM, Solbrig,Jeremy <
> > Jeremy.Solbrig at colostate.edu> wrote:
> > > Hello,
> > > 
> > > Much of what is below was copied from this stack overflow
> > > question.
> > > I am attempting to subclass numpy.ma.MaskedArray.  I am currently
> > > using Python v2.7.10.  The problem discussed below does not occur
> > > in Numpy v1.9.2, but does occur in all versions of Numpy v1.10.x.
> > > In all versions of Numpy v1.10.x, using mathematical operators on
> > > my subclass behaves differently than using the analogous ufunc.
> > > When using the ufunc directly (e.g. np.subtract(arr1,
> > >  arr2)), __array_prepare__, __array_finalize__, and
> > > __array_wrap__ are all called as expected, however, when using
> > > the symbolic operator (e.g. arr1-arr2) only __array_finalize__ is
> > > called. As a consequence, I lose any information stored in
> > >  arr._optinfo when a mathematical operator is used.
> > > Here is a code snippet that illustrates the issue.
> > > 
> > > #!/bin/env pythonimport numpy as np
> > > from numpy.ma import MaskedArray, nomask
> > > 
> > > class InfoArray(MaskedArray):
> > >     def __new__(cls, info=None, data=None, mask=nomask,
> > > dtype=None, 
> > >                 copy=False, subok=True, ndmin=0, fill_value=None,
> > >                 keep_mask=True, hard_mask=None, shrink=True,
> > > **kwargs):
> > >         obj = super(InfoArray, cls).__new__(cls, data=data,
> > > mask=mask,
> > >                       dtype=dtype, copy=copy, subok=subok,
> > > ndmin=ndmin, 
> > >                       fill_value=fill_value, hard_mask=hard_mask,
> > >                       shrink=shrink, **kwargs)
> > >         obj._optinfo['info'] = info
> > >         return obj
> > > 
> > >     def __array_prepare__(self, out, context=None):
> > >         print '__array_prepare__'
> > >         return super(InfoArray, self).__array_prepare__(out,
> > > context)
> > > 
> > >     def __array_wrap__(self, out, context=None):
> > >         print '__array_wrap__'
> > >         return super(InfoArray, self).__array_wrap__(out,
> > > context)
> > > 
> > >     def __array_finalize__(self, obj):
> > >         print '__array_finalize__'
> > >         return super(InfoArray, self).__array_finalize__(obj)if
> > > __name__ == "__main__":
> > >     arr1 = InfoArray('test', data=[1,2,3,4,5,6])
> > >     arr2 = InfoArray(data=[0,1,2,3,4,5])
> > > 
> > >     diff1 = np.subtract(arr1, arr2)
> > >     print diff1._optinfo
> > > 
> > >     diff2 = arr1-arr2
> > >     print diff2._optinfo
> > > If run, the output looks like this:
> > > 
> > > $ python test_ma_sub.py 
> > > #Call to np.subtract(arr1, arr2) here
> > > __array_finalize__
> > > __array_finalize__
> > > __array_prepare__
> > > __array_finalize__
> > > __array_wrap__
> > > __array_finalize__
> > > {'info': 'test'}#Executing arr1-arr2 here
> > > __array_finalize__
> > > {}
> > > Currently I have simply downgraded to 1.9.2 to solve the problem
> > > for myself, but have been having difficulty figuring out where
> > > the difference lies between 1.9.2 and 1.10.0.
> > > 
> > I don't see a difference between 1.9.2 and 1.10.0 in this test, so
> > I suspect it is something else. Could you try 1.10.4 to see if the
> > something else has been fixed?
> > 
> Which is to say, it isn't in the calls to prepare, wrap, and
> finalize. Now to look in _optinfo.
> 
> Chuck 
>  _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> https://mail.scipy.org/mailman/listinfo/numpy-discussion
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20160126/12059380/attachment.sig>