copy on write

[Steven D'Aprano]

On Fri, 03 Feb 2012 14:08:06 +1100, John O'Hagan wrote:

> I think we're 12 years late on this one. It's PEP 203 from 2000 and the
> key phrase was:
> 
> "The in-place function should always return a new reference, either to
> the old `x' object if the operation was indeed performed in-place, or to
> a new object."
> 
> If this had read:
> 
> "The in-place function should return a reference to a new object if the
> operation was not performed in-place."
> 
> or something like that, we wouldn't be discussing this.

And what should it return if the operation *is* performed in-place? 
"Don't return anything" is not an option, Python doesn't have procedures. 
That implies that __iadd__ etc. should return None. But two problems come 
to mind:

1) Using None as an out-of-band signal to the interpreter to say "don't 
perform the assignment" makes it impossible for the augmented assignment 
method to return None as the result. If we only think about numeric 
operations like x += 1 then we might not care, but once you consider the 
situation more widely the problem is clear:

x = Fact(foo)
y = Fact(bar)
x & y  # Returns a composite Fact, or None if they are contradictory

With your suggestion, x &= y fails to work, but only sometimes. And when 
it fails, it doesn't fail with an explicit exception, but silently fails 
and then does the wrong thing. This makes debugging a horror.

2) And speaking of debugging, sometimes people forget to include the 
return statement in methods. Normally, the left hand side of the 
assignment then gets set to None, and the error is pretty obvious as soon 
as you try to do something with it. But with your suggestion, instead of 
getting an exception, it silently fails, and your code does the wrong 
thing.

I suppose that they could have invented a new sentinel, or a special 
exception to be raised as a signal, but that's piling complication on top 
of complication, and it isn't clear to me that it's worth it for an 
obscure corner case.

Yes, the current behaviour is a Gotcha, but it's a Gotcha that makes good 
sense compared to the alternatives.

Ultimately, augmented assignment is *assignment*, just like it says on 
the tin. t[1] += x is syntactic sugar for t[1] = t[1].__iadd__(x). It 
can't and shouldn't fail to raise an exception if t is a tuple, because 
tuple item assignment *must* fail.

The problem is that lists treat __iadd__ as an in-place optimization, and 
this clashes with tuple immutability. But if lists *didn't* treat 
__iadd__ as in-place, people would complain when they used it directly 
without a tuple wrapper. 

Perhaps lists shouldn't define += at all, but then people will complain 
that mylist += another_list is slow. Telling them to use mylist.extend 
instead just makes them cranky. After all, mylist + another_list works, 
so why shouldn't += work?

Ultimately, there is no right answer, because the multitude of 
requirements are contradictory. No matter what Python did, somebody would 
complain.



-- 
Steven