Nested scopes, and augmented assignment

[Piet van Oostrum]

>>>>> Antoon Pardon <apardon at forel.vub.ac.be> (AP) wrote:

>AP> Well if someone explains what is wrong about my understanding, I
>AP> certainly care about that (although I confess to sometimes being
>AP> impatient) but someone just stating he is not sure I understand?

That is just a euphemistic way of stating `I think you do not understand
it'. 

>AP> So if I understand you correctly in code like:

>AP>   c.d = a
>AP>   b = a

>AP> All three names are searched for in all scopes between the local en global
>AP> one. That is what I understand with your statement that [python] always
>AP> uses the same search order.

The d is different because it is an attribute. So it is looked up in the
context of the object that is bound to c. For a, b, and c it is correct.

>AP> My impression was that python will search for c and a in the total current
>AP> namespace but will not for b.

The assignment to b inside the function (supposing the code above is part
of the function body) tells the compiler that b is a local variable. So the
search stops in the local scope. The search order is always from local to
global. First the current function, then nested function, then the module
namespace, and finally the builtins. The first match will stop the search.
Now for local variables inside a function, including the parameters, the
compiler will usually optimize the search because it knows already that it
is a local variable. But that is an implementation detail.

>>> But a variable that is bound
>>> inside the function (with an asignment) and is not declared global, is in
>>> the local namespace.

>AP> Aren't we now talking about implementation details? Sure the compilor
>AP> can set things up so that local names are bound to the local scope and
>AP> so the same code can be used. But it seems somewhere was made the
>AP> decision that b was in the local scope without looking for that b in
>AP> the scopes higher up.

Yes, as I (and others) have already said several times: an assignment to a
variable inside a function body (but not an assignment to an attribute or
part of an object) without a global declaration makes that variable a local
variable. That is not an implementation detail; it is part of the language definition.

>AP> Let me explain a bit more. Suppose I'm writing a python interpreter
>AP> in python. One implemantation detail is that I have a list of active
>AP> scopes which are directories which map names to objects. At the
>AP> start of a new function the scope list is adapted and all local
>AP> variables are inserted to the new activated scope and mapped to
>AP> some "Illegal Value" object. Now I also have a SearchName function
>AP> that will start at the begin of a scope list and return the
>AP> first scope in which that name exists. The [0] element is the
>AP> local scope. Now we come to the line "b = a"

>AP> This could be then executed internally as follows:

>AP>   LeftScope = SearchName("b", ScopeList)
>AP>   RightScope = SearchName("a", ScopeList)
>AP>   LeftScope["b"] = RightScope["a"]

>AP> But I don't have to do it this way. I already know in which scope
>AP> "b" is, the local one, which has index 0. So I could just as well
>AP> have that line exucuted as follows:

>AP>   LeftScope = ScopeList[0]
>AP>   RightScope = SearchName("a", ScopeList)
>AP>   LeftScope["b"] = RightScope["a"]

>AP> As far as I understand both "implementations" would make for
>AP> a correct execution of the line "b = a" and because of the
>AP> second possibility, b is IMO not conceptually searched for in
>AP> the same way as a is searched for, although one could organise
>AP> things that the same code is used for both.

That is the optimization I spoke of above. But it is not the problem we
were discussing. Conceptually it is the same. It is similar to constant
folding (replacing x = 2+3 by x = 5).

>AP> Of course it is possible I completely misunderstood how python
>AP> is supposed to work and the above is nonesense in which case
>AP> I would appreciate it if you correct me.

>AP> Python could have chosen an approach with a "nested" keyword, to allow
>AP> rebinding a name in an intermediate scope. It is not that big a deal
>AP> that it hasn't, but I keep finding the result strange and somewhat
>AP> counterintuitive.
>>> 
>>> Maybe it would have been nice if variables could have been declared as
>>> nested, but I think it shows that nested variables have to be used with
>>> care, similar to globals. Especially not allowing rebinding in intermediate
>>> scopes is a sound principle (`Nested variables considered harmful').
>>> If you need to modify the objects which are bound to names in intermediate
>>> scopes, use methods and give these objects as parameters.

>AP> But shouldn't we just do programming in general with care? And if
>AP> Nested variables are harmfull, what is then the big difference
>AP> between rebinding them and mutating them that we should forbid
>AP> the first and allow the second?

There is no big difference I think. Only Python doesn't have syntax for the
former. Older versions of Python didn't even have nested scopes. maybe it
was a mistake to add them. I think an important reason was the use in
lambda expressions, which Guido also regrets IIRC.

>AP> I understand that python evolved and that this sometimes results
>AP> in things that in hindsight could have been done better. But
>AP> I sometimes have the impression that the defenders try to defend
>AP> those results as a design decision. With your remark above I have
>AP> to wonder if someone really thought this through at design time
>AP> and came to the conclusion that nested variables are harmfull
>AP> and thus may not be rebound but not that harmfull so mutation
>AP> is allowed and if so how he came to that conclusion.

I don't think that was the reasoning. On the other hand I think nested
scopes were mainly added for read-only access (to the namespace that is,
not to the values) . But then Python doesn't forbid you to change mutable
objects once you have access to them.
-- 
Piet van Oostrum <piet at cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: piet at vanoostrum.org