Scope rule pecularities

[Andrew Bennetts]

On Thu, May 13, 2004 at 11:26:28AM +0000, Antoon Pardon wrote:
> Op 2004-05-13, Andrew Bennetts schreef <andrew-pythonlist at puzzling.org>:
> >
> > Well, classes get to do whatever they think makes sense: in the end it's all
> > calls to methods of the object, whether via an actual method call
> > "obj.foo()" or by an operator "obj * 2" (which calls obj.__mul__(2)) or an
> > augmented assignment "obj += 'x'" (which calls obj.__iadd__('x')).  It's up
> > to the objects to make sense, not the Python language.
> 
> And how do you think objects can make sense where the language doesn't?
> I don't want to imply the language doesn't make sense at all, but you
> can't claim the responsibility is all for the objects if the language
> or its core classes have defiencies or are not consistent with each
> other.

I'm saying that in this case, the objects have a responsibility to make
sense, and this is necessarily true in any language that allows operator
overloading.  If an object insists on always returning 7 regardless of what
is added to it, then there's not much hope that the '+' operator will do
what users expect.

> > Even though Python chooses to have some immutable builtin types (like
> > ints and strings) for a variety of practical reasons, augmented assignment
> > does what people expect in all these cases:
> >
> >     s = 'abc'
> >     s += 'd'
> >
> >     i = 7
> >     i += 3
> >
> >     l = [1, 2, 3]
> >     l += [4, 5, 6]
> >
> > Augmented assignments are still assignments, and that makes perfect sense to
> > me -- each of those behave exactly like their obvious longer versions:
> >
> >     s = 'abc'
> >     s = s + 'd'
> >
> >     i = 7
> >     i = i + 3
> >
> >     l = [1, 2, 3]
> >     l = l + [4, 5, 6]
> >
> > (Yes, there are tricky examples that do behave differently -- but I try to
> > avoid tricky things, because they tend to be hard to read.  I almost only
> > find I want to use augmented assignment on integers and occasionally
> > strings.)
> 
> But these things are only hard to read (to you, I don't find it so) because
> you are using constants in the expression.

I didn't say those examples are hard to read.  Personally, I find both
forms equally readable.

I did say that there are cases where:

    <lvalue> += x

does not behave the same way as:

    <lvalue> = <lvalue> + x

But you need relatively tricky lvalues for this to be the case.

> a += b, is just as hard to read whether a and b are lists or integers.
> 
> The problem is that because the behaviour with strings and lists is
> different I can't write a program in which the behaviour of += is
> consistent for all classes because there will always be core classes
> for which the behaviour will be different.

It seems to be that your problem isn't really with "+=" as such, it's that
Python has immutable types at all, including in the commonly used builtins.
Immutability of strings, numbers and tuples has worked very well for Python
so far, so you'll have a struggle convincing many people that it should be
otherwise.

Personally, I like that when I do:

    x = 7
    func(x)

That I know that x will still be 7 after func returns, regardless of what
happens inside func.  It's just not possible for func to accidentally change
the value of x in my scope, because I know that numbers are immutable in
Python.

It would take some *massive* benefits to convince me to change my mind.

> It is a bit like having some classes use "+" for substraction. In it self
> that wouldn't be so bad, you just have to pay attention. But then
> you want to write a class that will work with whatever number type
> and it needs to do additions. Now suddenly things get difficult because
> a + b doesn't behave consistently among the different number types. 

Again, if the objects you're using don't "make sense", you're stuck with a
difficult life.  If a library's API is hard to use, then chances are the
library won't get used much.

> You have the same kind of difficulty now if you want to write a
> class/function/module that works with any kind of sequence.
> 
> Using the "+=" operator you can't guarantee any kind of consistency.
> If I had code like this:
> 
>   a = b
>   b += c
> 
> I would have no idea at all whether a was changed or not. So
> writing code that can work with any sequence becomes a possible
> problem spot if you use these kind of operators.

Then take a copy of the sequence first, using list(seq) or tuple(seq), and
then you'll know exactly what you have.  Or even simpler -- use: "b = b + c"
instead.  Then you know that a is unmodified.

If you don't know what kind of objects your functions are working with, how
can you expect them to behave correctly?  If your API assumes that a
particular method receives a tuple, then either make sure it does by calling
tuple(x), or document that it does in the docstring.

Otherwise, a user of your library code might legitimately try to pass a
list, a dictionary, None, or perhaps their own custom type, unaware of your
hidden assumptions.

In my experience, I've never had the issue of mutable vs. immutable sequence
types cause the sort of problems you're worried about.  Perhaps I'm lucky,
or subconciously careful, or it has happened and I'm just plain forgetful :)
... Regardless, I'm not going to start worrying about it now.  Python is
working just fine for me despite this.

-Andrew.