[Numpy-discussion] possible bug: __array_wrap__ is not called during arithmetic operations in some cases

[Darren Dale]

On Sun, Mar 8, 2009 at 4:54 PM, Charles R Harris
<charlesr.harris at gmail.com>wrote:

>
>
> On Sun, Mar 8, 2009 at 2:48 PM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Sun, Mar 8, 2009 at 1:04 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>>
>>> On Sat, Mar 7, 2009 at 1:23 PM, Darren Dale <dsdale24 at gmail.com> wrote:
>>>
>>>> On Sun, Feb 22, 2009 at 7:01 PM, Darren Dale <dsdale24 at gmail.com>wrote:
>>>>
>>>>> On Sun, Feb 22, 2009 at 6:35 PM, Darren Dale <dsdale24 at gmail.com>wrote:
>>>>>
>>>>>> On Sun, Feb 22, 2009 at 6:28 PM, Pierre GM <pgmdevlist at gmail.com>wrote:
>>>>>>
>>>>>>>
>>>>>>> On Feb 22, 2009, at 6:21 PM, Eric Firing wrote:
>>>>>>>
>>>>>>> > Darren Dale wrote:
>>>>>>> >> Does anyone know why __array_wrap__ is not called for subclasses
>>>>>>> >> during
>>>>>>> >> arithmetic operations where an iterable like a list or tuple
>>>>>>> >> appears to
>>>>>>> >> the right of the subclass? When I do "mine*[1,2,3]", array_wrap is
>>>>>>> >> not
>>>>>>> >> called and I get an ndarray instead of a MyArray. "[1,2,3]*mine"
>>>>>>> is
>>>>>>> >> fine, as is "mine*array([1,2,3])". I see the same issue with
>>>>>>> >> division,
>>>>>>> >
>>>>>>> > The masked array subclass does not show this behavior:
>>>>>>>
>>>>>>> Because MaskedArray.__mul__ and others are redefined.
>>>>>>>
>>>>>>> Darren, you can fix your problem by redefining MyArray.__mul__ as:
>>>>>>>
>>>>>>>     def __mul__(self, other):
>>>>>>>         return np.ndarray.__mul__(self, np.asanyarray(other))
>>>>>>>
>>>>>>> forcing the second term to be a ndarray (or a subclass of). You can
>>>>>>> do
>>>>>>> the same thing for the other functions (__add__, __radd__, ...)
>>>>>>
>>>>>>
>>>>>> Thanks for the suggestion. I know this can be done, but ufuncs like
>>>>>> np.multiply(mine,[1,2,3]) will still not work. Plus, if I reimplement these
>>>>>> methods, I take some small performance hit. I've been putting a lot of work
>>>>>> in lately to get quantities to work with numpy's stock ufuncs.
>>>>>>
>>>>>
>>>>> I should point out:
>>>>>
>>>>> import numpy as np
>>>>>
>>>>> a=np.array([1,2,3,4])
>>>>> b=np.ma.masked_where(a>2,a)
>>>>> np.multiply([1,2,3,4],b) # yields a masked array
>>>>> np.multiply(b,[1,2,3,4]) # yields an ndarray
>>>>>
>>>>>
>>>> I'm not familiar with the numpy codebase, could anyone help me figure
>>>> out where I should look to try to fix this bug? I've got a nice set of
>>>> generators that work with nosetools to test all combinations of numerical
>>>> dtypes, including combinations of scalars, arrays, and iterables of each
>>>> type. In my quantities package, just testing multiplication yields 1031
>>>> failures, all of which appear to be caused by this bug (#1026 on trak) or
>>>> bug #826.
>>>
>>>
>>>
>>> I finally managed to track done the source of this problem.
>>> _find_array_wrap steps through the inputs, asking each of them for their
>>> __array_wrap__ and binding it to wrap. If more than one input defines
>>> __array_wrap__, you enter a block that selects one based on array priority,
>>> and binds it back to wrap. The problem was when the first input defines
>>> array_wrap but the second one does not. In that case, _find_array_wrap never
>>> bothered to rebind the desired wraps[0] to wrap, so wrap remains Null or
>>> None, and wrap is what is returned to the calling function.
>>>
>>> I've tested numpy with this patch applied, and didn't see any
>>> regressions. Would someone please consider committing it?
>>>
>>> Thanks,
>>> Darren
>>>
>>> $ svn diff numpy/core/src/umath_ufunc_object.inc
>>> Index: numpy/core/src/umath_ufunc_object.inc
>>> ===================================================================
>>> --- numpy/core/src/umath_ufunc_object.inc       (revision 6569)
>>> +++ numpy/core/src/umath_ufunc_object.inc       (working copy)
>>> @@ -3173,8 +3173,10 @@
>>>              PyErr_Clear();
>>>          }
>>>      }
>>> +    if (np >= 1) {
>>> +        wrap = wraps[0];
>>> +    }
>>>      if (np >= 2) {
>>> -        wrap = wraps[0];
>>>          maxpriority = PyArray_GetPriority(with_wrap[0],
>>>                                          PyArray_SUBTYPE_PRIORITY);
>>>          for (i = 1; i < np; ++i) {
>>>
>>
>> Applied in r6573. Thanks.
>>
>
> Oh, and can you provide a test for this fix?
>

Yes, I'll send a patch for a test as soon as its ready. 6573 closes two
tickets, 1026 and 1022. Did you see the patch I sent for issue #826? It is
also posted at the bug report.

Darren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20090308/118971bf/attachment.html>