Nested scopes and lexical closures

[Alex Martelli]

"Paul Rubin" <phr-n2001 at nightsong.com> wrote in message
news:7xu1z5c9lk.fsf at ruckus.brouhaha.com...
> I see that Python 2.1.1 lets you import nested scopes from the future.
> However, I couldn't easily tell from the documentation whether that
> means you can make lexical closures:

Yes, but not re-bind variables bound in enclosing blocks.


> Of course that's disappointing and I'm wondering if there's some
> intention to fix it in the future.

Only Guido knows, but I suspect he doesn't mean to.  But if you
can think of an utterly simple and obvious syntax that lets you
bind a variable locally even when it's already bound in an
enclosing block, AND also lets you re-bind the variable in the
enclosing block, he might be interested -- I think the inability
to find a syntax to express both in obvious ways was a decisive
factor.

I thought the Java rule might be better: when a variable named
X is bound in an enclosing block, an inner block CANNOT define
a variable named X -- neither bind a new one nor rebind the
one in the outer block.  This would not change functionality
(you'd just have to change names of new variables in inner
blocks whose names conflict with outer-block variables), but
I thought it would avoid the slight confusion that comes with
the present situation, where one THINKS one's re-binding an
outer-block variable but is in fact binding a new local homonym
that shadows (hides) the outer one.

Java adopted this restrictive rule because, in their opinions
and/or studies, homonym variables between inner and outer
blocks turned out to be errors much more often than they
were deliberate choices -- and even when deliberate, the
resulting program was clearer if the inner variable had to
be renamed.  Of course, Java and most other languages are
different from Python in that they distinguish definition and
assignment, which Python doesn't (we only have binding
and re-binding of variables, not 'assignment' as in most
other languages, although we use assignment-syntax).

Anyway, I don't think my musings of the time were very
much listened to.  Some people can't stand my style, and,
alas, it appears our beloved BDFL belongs to that unlucky
minority (not too sure it IS a minority, but they're surely
unlucky for their unfortunate dislikes, aren't they:-).


> Meanwhile I think the
> documentation of nested scopes should be updated to make it clear that
> closures don't work.

Let's see, Python 2.1 reference manual, Appendix A.3 "Nested Scopes",
section A.3.1:

"""
When a name is used in a code block, it is resolved using the nearest
enclosing
scope. The set of all such scopes visible to a code block is called the
block's
environment.

If a name is bound in a block, it is a local variable of that block.
    ...
If a name binding operation occurs anywhere within a code block, all uses of
the name within the block are treated as references to the current block.
This
can lead to errors when a name is used within a block before it is bound.

The previous rule is a subtle. Python lacks declarations and allows name
binding operations to occur anywhere within a code block. The local
variables
of a code block can be determined by scanning the entire text of the block
for
name binding operations.
"""

It seems reasonably clear to me.

PEP 227 at http://python.sourceforge.net/peps/pep-0227.html is
I think even clearer:
"""
As a
    consequence, it is not possible to rebind a name defined in an
    enclosing scope.  An assignment operation can only bind a name in
    the current scope or in the global scope.  The lack of
    declarations and the inability to rebind names in enclosing scopes
    are unusual for lexically scoped languages; there is typically a
    mechanism to create name bindings (e.g. lambda and let in Scheme)
    and a mechanism to change the bindings (set! in Scheme).

    XXX Alex Martelli suggests comparison with Java, which does not
    allow name bindings to hide earlier bindings.
"""

What "documentation of nested scopes" is so unclear and needs
to add copies or paraphrases of this paragraph?  If you can supply
the URL's, I'm sure Fred Drake will be happy to fix them as soon
as somebody opens a bug report on the relevant docs.

Note that the fact that you can't re-bind names in enclosing
scopes doesn't mean "you can't make lexical closures" -- there
are plenty of functional languages that are single-assignment
and use immutable data (you can never re-bind any name
anywhere [nor are data mutable]), and yet they make plenty
of lexical closures and use them abundantly.  Python's lexical
closures are more fluid than those, since you CAN still rebind
a local or module-global name, AND mutate (just not re-bind)
data that's bound to enclosing-scope names.


Alex