[Python-Dev] Lexical scoping in Python 3k

[Talin]

Josiah Carlson wrote:
> I had hoped that there would be a response to my second (and I believe
> more applicable statement); "if the feature is really only useful for
> generally trivial cases *without* the feature, then making them even
> more trivial, I think, is a bit of over optimization."

It really depends on how common the trivial case is. In other words, 
multiply the savings for each occurance times the number of occurances. 
(Unfortunately, I don't know what units to measure said savings in - is 
there a unit of 'mental disconnect' or unintuitiveness? :)

In an idealy world, the language would allow everything to be said in 
the most comprehensible way possible. Longer and more verbose ways of 
stating something are at an inherent disadvantage in this, simply 
because of the time it takes to scan and absorb the information by the 
human brain. However, losing excess syntax has to be done in a way that 
doesn't also lose information. Highly compressed representations of a 
concept may require such a level of abstraction that it is as much work 
to puzzle out their meaning as it would be to read the longer version 
and more.

To put it another way - I am an advocate of applying Claude Shannon's 
theory of information to language design. The highest level of 
compression should be used for expressions that occur the most frequently.

> As for a solution, I find the "global means 'not local'" proposition is
> the least undesireable of the possibilities.  It suffers from a change
> in semantics and potential name masking issues, but I don't believe
> these are any more serious than normal global masking for the former,
> and the latter is solvable with a __future__ (at least for 2.6). I'm a
> solid -0 on this particular proposition, which is far better than the -1
> I am on all of the other recent lexical scoping propositions.

I'd say that the more common case is where you want global to really 
mean global - that is, you want to be able to write to some module-level 
variable, regardless of how deeply nested your function scope is. While 
being able to access the 'next outer scope' is occasionally useful, it's 
not all that common. So changing the behavior of 'global' in this case 
would be both confusing (since it no longer means 'global'), and less 
useful (because it doesn't match the most common case.)

(This assumes that I haven't completely understood the meaning of the 
phrase 'not local' - I assumed that it means 'not defined in this scope')

Of course, the reason why it's not all that common may be because of the 
fact that it's not as easy to do, and so people tend to (consciously or 
otherwise) avoid that pattern in their designs. That being said, I don't 
think that's necessarily such a bad thing. Python isn't Scheme, and the 
scoping rules of Python are IMHO more oriented towards practicality and 
common sense than theoretical purity.

This is why I'm not bothered by the fact that Python doesn't create a 
new scope for loop statements and such. Most of the time, this is what 
you want. It does mean that you need to name all of your variables 
uniquely, but that's good programming style in any case. The same is 
true for local variables not needing to be specially declared as 'my' or 
'var' - most of the time, a local variable is what you want.

On the other hand, the thing about theoretical purity is that it can be 
so mouth-wateringly powerful at times. For example, a language that 
supports closures is, IMHO, at least twice as powerful as a language 
that doesn't -- because you can use them in so many different and 
interesting ways.

OK, so about the lexical scoping issue - let me brainstorm a moment:

One idea would be to introduce the keyword 'local' which would have the 
effect of capturing any 'global' statements in any enclosing scope. So 
for example:

    f = 1
    def a():
       local f
       f = 2
       def b():
          global f
          f = 3

So in this case, the 'global' statement, which would normally associate 
'f' with the outermost (module-level) scope, would instead associate 'f' 
with the innermost 'local' declaration of that variable. So in the above 
example, assigning 3 to f assigns it to the middle scope, but does not 
affect the module-level definition.

Admittedly, that's a bit confusing and also verbose, considering that 
you are not only adding an extra keyword, but also using two statements 
to specify the home of one variable.

Another alternative would be a way to declare an explicitly scoped 
variable. Lets use the keyword 'my' to indicate this:

    f = 1
    def a():
       my f = 2
       def b():
          f = 3

In this case, what the 'my' statement is doing is indicating that this 
scope 'owns' the definition of 'f' -- in other words, the definition is 
hoisted out of any enclosed scopes. So again, in the above example, the 
innermost assignment will be to the definition of 'f' in the middle scope.

What's interesting about this is that you can use the same method with 
globals:

    my f = 1
    def a():
       f = 2

    a()
    print f   # prints '2'

So again, you are indicating that the global scope 'owns' the definition 
of 'f', and any enclosed scopes should use that definition, and not 
create their own.

Of course, if you really *do* need to have your own version, you can 
always override the 'my' statement with another 'my' statement:

    my f = 1
    def a():
       my f = 2

    a()
    print f   # prints '1'

The 'my' statement essentially changes the scoping rules for all 
variables of that name, within the defining scope and all enclosed scopes.

Of course, you can also override this behavior using the 'global' 
statement, which does exactly what it does now - makes the reference 
global (i.e. module-level):

    my f = 1
    def a():
       global f
       f = 2

    a()
    print f   # prints '2'

All right, I'm pretty happy with that. Brainstorming done. :)

-- Talin