[Python-Dev] behavior of inplace operations

[David Abrahams]

From: "Guido van Rossum" <guido@python.org>


> > My initial post asking about the implementation of += sparked a
> > small thread over on python-list, from which I've come to the
> > conclusion that my little optimization suggestion (don't try to set
> > the attribute or item if the inplace op returns its first argument)
> > is actually more semantically correct.
> [...]
> > Thoughts?
>
> One problem is that it's really hard to design the bytecode so that
> this can be implemented.  The problem is that the compiler sees this:
>
>    a[i] += x
>
> and must compile bytecode that works for all cases: a can be mutable
> or immutable, and += could return the same or a different object as
> a[i].  It currently generates code that uses a STORE_SUBSCR opcode
> (store into a[i]) with the saved value of the object and index used
> for the BINARY_SUBSCR (load from a[i]) opcode.  It would have to
> generate additional code to (a) save the object retrieved from a[i]

Isn't that already lying about on the stack somewhere? Didn't you have to
have it in order to invoke "+= x" on it? (I'm totally ignorant of Python's
bytecode, I'll be the first to admit)

> (b) compare the result to it using the 'is' operator, and (c) pop some
> stuff of the stack and skip over the assignment if true.  That could
> be done, but the extra test would definitely slow things down.

As was suggested by someone else in the thread I referenced, I was thinking
that a new bytecode would be used to handle this. It has to be faster to do
one test in 'C' code than it is to re-indexing into a map or even to do the
refcount-twiddling that goes with an unneeded store into a list.

> A worse problem is that it's a semantic change.  For example,
> persistent objects in Zope require (under certain circumstances) that
> if you modify an object that lives in a persistent container, you have
> to store it back into the container in order for the persistency
> mechanism to pick up on the change.  Currently we can rely on a[i]+=x
> and a.foo+=x to do the assigment.  Under your proposal, we couldn't
> (unless we knew that the item was of an immutable type).

That's right. I would have suggested that for persistent containers, the
object returned carries its own write-back knowledge.

> That is such
> a subtle change in semantics that I don't want to risk it without
> years of transitional warnings.

Hah, code breakage. The purity of the language must not be compromised, at
any cost! Well, ok, if someone's actually using this extra step I guess you
can't change it on a whim...

> Personally, I'd rather accept that if you have a = ([], [], []),
> a[1]+=[2] won't work.  You can always write a[1].extend([2]).

It's your choice, of course. However, it seems a little strange to have
this fundamental operation which is optimized for persistent containers but
doesn't work right -- and (I assert without evidence) must be slower than
neccessary -- in some simple cases. The pathological/non-generic cases are
the ones that make me think twice about using the inplace ops at all. They
don't, in fact, "just work", so I have to think carefully about what's
happening to avoid getting myself in trouble.

-Dave