[Python-Dev] LOAD_NAME & classes

[Guido van Rossum]

[Tim]
> There's a point to that too: uniqueness also imposes costs on
> newbies and/or newcomers.  Across the world of programming languages
> now, dynamic scoping and lexical scoping are "almost entirely *it*".
[...]
> Nor its advantages, including better error detection, and ease of
> transferring hard-won knowledge among other lexically scoped
> languages.

But Python *is* unique in that it doesn't require declarations.  (I've
got to admit that the Perl example saddened me.  But then in Perl,
local variables are a recent invention. :-)

We've found before that this can go against what's common knowledge
for other language (e.g. integer division).

[...]
> Does it scale?
> 
> x = 0
> 
> def f(i):
>     if i & 4:
>         x = 10
>     def g(i):
>         if i & 2:
>             x = 20
>         def h(i):
>             if i & 1:
>                 x = 30
>             print x
>         h(i)
>     g(i)
> 
> f(3)
> 
> I can look at that today and predict with confidence that h() will
> either print 30 (if and only if i is odd), or raise an exception.
> This is from purely local analysis of h's body -- it doesn't matter
> that it's nested, and it's irrelvant what the enclosing functions
> look like or do.  That's a great aid to writing correct code.  If
> the value of x h sees *may* come from h, or from g, or from f, or
> from the module scope instead, depending on i's specific value at
> the time f is called, there's a lot more to think about.

Yup.  But it also requires intricate knowledge of Python's rules,
which are different than any other language's rules.  You simply can't
have a variable declaration inside an if statement in other languages
that extends to the entire function body -- either the scope would be
limited to the block it's in, or the syntax wouldn't allow it.

Python's original semantic model on the other hand, and the model
that's still used for globals at the global level, gives a clear
explanation: a namespace is implemented as a dictionary, and name
lookup searches a pre-set sequence of namespaces until it finds a hit.
The lexical scoping rule determines how namespaces are combined.
Doing the lookup at runtime is easier to understand than doing it at
compile time -- even if the compile version might catch more bugs.
But I'm repeating myself; I already said that in my previous message.

> I could keep local+global straight in pre-1.0 Python, although I
> never got used to the inability to write nested functions that could
> refer to each other (perhaps you've forgotten how many times you had
> to explain that one, and how difficult it was to get across?).

No; apart from you, most people were happy with the rule "nested
functions don't work".

> Now that Python has full-blown nested scopes, the namespace
> interactions are potentially much more convoluted, and the "purely
> local analysis" shortcut made possible by everyone else's <wink>
> notion of lexical scoping becomes correspondingly more valuable.

I don't know.  Full-blown nested scopes make namespace interactions
more convoluted no matter *what* set of rules we pick.  An alternative
implementation model (with associated subtly different semantics
semantics) would have been to create an explicit list of the dicts
involved in the name resolution for a particular function invocation;
we rejected that model because we wanted this to be (nearly) as fast
as locals, so we moved more of the analysis to compile time.

But by doing so, we introduced more of a dependency on the
programmer's ability to understand what happens at compile time, and
that breaks the "only runtime exists" illusion.

In PEP 267 Jeremy is exploring how to optimize access to globals
*without* changing the rules.  The change to LOAD_FAST that I
considered before would have optimized access to locals without
changing the rules, and I still regret that I didn't think of that
when I created LOAD_FAST (even though you disagree): the "only
runtime" rule is helpful for a large class of programmers, not only
newbies, and I'm not sure that adding more and more cruft from truly
compiled languages to Python's *semantics* is a good idea.  (Adding
compiler technology that doesn't change the rules is fine, of course,
if it helps optimizations or better diagnostics.)

> > ...
> > Um, that's not what I'd call dynamic scoping.  It's dynamic lookup.
> 
> I know -- the problem is that you're the only one in the world
> making this distinction, and that makes it hard to maintain over
> time.

You can say that, but that doesn't make it so, and it doesn't convince
me.  The three-scope was gospel in the Python world, and many people
actively disliked adding nested scopes (some still do).

> If it had some killer advantage ... but it doesn't seem to.
> When Python switched to "strict local" names before 1.0, I don't
> recall anyone complaining -- if there was a real advantage to
> dynamic lookup at the local scope, it appeared to have escaped
> Python's users <wink>.  I'll grant that it did make exec and "import
> *" more predictable in corner cases.

Well, we gave them a big reason not to complain: this was the
singlemost biggest speedup in Python's history.  But the rules were
definitely harder to explain, because for the first time we had to
explain a second compiler pass.

> > It's trouble for a compiler that wants to optimize builtins, but the
> > semantic model is nice and simple and easy to explain with the "only
> > runtime" rule.
> 
> Dynamic scoping is also easy to explain, but it doesn't scale.  I'm
> afraid dynamic lookup doesn't scale either.  You should have stuck
> with Python's original two-level namespace, you know <0.9 wink>.

We need more than a single example to decide which rules bites worse
for large programs.  Deep nesting is not common; long functions are.
And there the common annoyance is that a change in line 150 can break
the code in line 2 of the function.

--Guido van Rossum (home page: http://www.python.org/~guido/)