[Python-Dev] INPLACE_ADD and INPLACE_MULTIPLY oddities in ceval.c

[Guido van Rossum]

On 3/27/06, Travis E. Oliphant <oliphant.travis at ieee.org> wrote:
> Guido van Rossum wrote:
> > On 3/27/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
> >> If you have Numeric or numpy installed try this:
> >>
> >> #import Numeric as N
> >> import numpy as N
> >>
> >> a = range(10)
> >> b = N.arange(10)
> >>
> >> a.__iadd__(b)
> >>
> >> print a
> >>
> >> Result:
> >>
> >> [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
> >>
> >>
> >> Contrast the returned output with
> >>
> >> import numpy as N
> >>
> >> a = range(10)
> >> b = N.arange(10)
> >>
> >> a += b
> >>
> >> print a
> >>
> >> Result:
> >>
> >> [ 0  2  4  6  8 10 12 14 16 18]
> >>
> >>
> >> Having "a+=b" and "a.__iadd__(b)" do different things seems like an
> >> unfortunate bug.
> >>
> >> It seems to me that the problem is that the INPLACE_ADD and
> >> INPLACE_MULTIPLY cases in ceval.c  use PyNumber_InPlace<YYY> instead of
> >> trying PySequence_InPlace<YYY> when the object doesn't support the
> >> in-place number protocol.
> >>
> >> I could submit a patch if there is agreement that this is a problem.
> >
> > Well how is the interpreter to know whether += meanse numeric add or
> > sequence add in any particular case?
>
> I can see that '+' (and '*') having two different implementation slots
> is a design bug.  However, it seems prudent that both += and .__iadd__
> should have identical behavior regardless.  Whatever mechanism is used
> to resolve the conflict for __iadd__, the same mechanism should be used
> for for +=.
>
> I don't think this is a NumPy-only issue.  Any object that defines
> addition that works with lists will have similar problems.
>
> I think this can be fixed easily by first checking the sequence slot for
> a sq_concat function before calling PyNumber_InPlaceAdd.
>
> All I'm asking for is that a += b  have the same behavior as
> a.__iadd__(b).   That seems like desireable behavior to me.

Correct. Robert Kern's analysis is correct: when you make the call
explicitly, a.__iadd__(b) invokes the list.__iadd__ method; this is
effectively an alias for list.extend. I take back whatever I said
about numpy.

So for consistency we want a += b to also execute a.__iadd__. The
opcode calls PyNumber_InplaceAdd; I think that PyNumber_InplaceAdd
(and PySequence_InplaceConcat, if it exists) should test for both the
numeric and the sequence augmented slot of the left argument first;
then they should try both the numeric and sequence non-augmented slot
of the left argument; and then the numeric non-augmented slot of the
right argument. Coercion should not be attempted at all.

The question is, can we do this in 2.5 without breaking backwards
compatibility? Someone else with more time should look into the
details of that.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)