[Python-Dev] Re: closure semantics

[Guido van Rossum]

> > or at run-time) always goes from inner scope to outer.  While you and
> > I see nested functions as small amounts of closely-knit code, some
> > people will go overboard and write functions of hundred lines long
> > containing dozens of inner functions, which may be categorized into
> 
> This doesn't look like a legitimate use case to me; i.e., I see no need
> to distort the language if the benefit goes to such "way overboard" uses.
> I think they will have serious maintainability problems anyway.

One person here brought up (maybe David Eppstein) that they used this
approach for coding up extensive algorithms that are functional in
nature but have a lot of state referenced *during* the computation.
Whoever it was didn't like using classes because the internal state
would persist past the lifetime of the calculation.

When I visited Google I met one person who was advocating the same
coding style -- he was adamant that if he revealed any internal
details of his algorithm then the users of his library would start
using them, and he wouldn't be able to change the details in another
revision.

AFACT these were both very experienced Python developers who had
thought about the issue and chosen to write large nested functions.

So I don't think you can dismiss this so easily.

> Fortunately, I don't think of placing the "indication to the
> compiler" as close to the assignment-to-outer-variable as a
> distortion;-)
> 
> > Anyway, I hope you'll have a look at my reasons for why the compiler
> > needs to know about rebinding variables in outer scopes from inside
> > an inner scope.
> 
> Sure!  I do understand this.  What I don't understand is why,
> syntactically, the reserved word that indicates this to the compiler
> should have to be a "statement that does nothing" -- the ONLY
> "declaration" in the language -- rather than e.g. an _operator_
> which specifically flags such uses.

Maybe because I haven't seen such an operator proposed that I
liked. :)

And in its normal usage, I don't find 'global x' offensive; that it
can be abused and sometimes misunderstood doesn't matter to me, that's
the case for sooooo many language constructs...

> Assume for the sake of argument that we could make 'scope' a reserved
> word.  Now, what are the tradeoffs of using a "declaration"
>     scope x in outer
> which makes all rebidings of x act in the scope of containing function
> outer (including 'def x():', 'class x:', 'import x', ...); versus an 
> "operator" that must be used to indicate "which x" when specifically
> assigning it (no "side effect rebinding" via def &c allowed -- I think it
> helps the reader of code a LOT to require specific assignment!), e.g.
>     scope(outer).x = 23
> 
> Don't think of scope as a built-in function, but as a keyword in either
> case (and we could surely have other syntax for the "scope operator",
> e.g. "(x in outer scope) = 23" or whatever, as long as it's RIGHT THERE
> where x is being assigned).  So the compiler can catch on to the info
> just as effectively.

What bugs me tremendously about this is that this isn't symmetric with
usage: you can *use* the x from the outer scope without using all that
verbiage, but you must *assign* to it with a special construct.  This
would be particularly confusing if x is used on the right hand side of
the assignment, e.g.:

  scope(outer).x = x.lower()

> The tradeoffs are:
>    -- we can keep thinking of Python as declaration-free and by gradually
>        deprecating the global statement make it more so

Somehow I don't see "declaration-free" as an absolute goal, where 100%
is better than 99%.

>    -- the reader of code KNOWS what's being assigned to without having
>        to scroll up "hundreds of lines" looking for possible declarations

Yeah, but you can still *use* a variable that was set "hundreds of
lines" before, so it's not a full solution (and will never be --
allowing *use* of nonlocals is clearly a much-wanted and very useful
feature).

>    -- assignment to nonlocals is made less casually convenient by just the
>        right amount to ensure it won't be overused

If we don't add "global x in f" or some equivalent, you can't assign
to nonlocals except for module globals, where I don't see a problem.

>    -- no casual rebinding of nonlocals via def, class, import

I don't think that's a real issue.

>    -- once we solve the general problem of allowing non-bare-names as
>        iteration variables in 'for', nonlocals benefit from that
>        resolution automatically, since nonlocals are never
>        assigned-to as bare-names

This is obscure -- most readers here didn't even know you could do
that, and all except Tim (whom I cut a certain amount of slack because
he's from Wisconsin) said they considered it bad style.  So again the
argument is weak.

> I see this as the pluses.  The minus is, we need a new keyword; but I
> think we do, because stretching 'global' to mean something that ISN'T
> global in any sense is such a hack.

Well, if for some reason the entire Python community suddenly leaned
on me to allow assignment to non-locals with a syntactic construct to
be used in every assignment to a non-local, I would much favor the C++
style of <scope>::<name>.

> Cutting both ways is the fact that this allows using the same name from
> more than one scope (since each use is explicitly qualified as coming
> from a specific scope).  That's irrelevant for small compact uses of
> nesting, but it may be seen as offering aid and succour to those wanting
> to "go overboard" as you detail earlier (bad);

There is no need for this even among those folks; a simple renaming
allows access to all variables they need.  (My earlier argument wasn't
about this, it was about accidental shadowing when there was *no* need
to share.)

> OTOH, if I ever need to maintain such "overboard" code written by
> others, and refactoring it is not feasible right now, it may be
> helpful.  In any case, the confusion is reduced by having the
> explicit qualification on assignment.  Similarly for _accesses_
> rather than rebindings -- access to the barename will keep using the
> same rules as today, of course, but I think the same syntax that
> MUST be used to assign nonlocals should also be optionally usable to
> access them -- not important either way in small compact functions,
> but more regular and offering a way to make code non-ambiguous in
> large ones.  I don't see having two ways to access a name --
> barename x or qualified scope(foo).x -- as a problem, just like
> today from inside a method we may access a classvariable as "self.x"
> OR "self.__class__.x" indifferently -- the second form is needed for
> rebinding and may be chosen for clarity in some cases where the
> first simpler ("barer") one would suffice.

Actually, self.__class__.x is probably a mistake, usually one should
name the class explicitly.

But I don't see that as the same, because the name isn't bare in
either case.

--Guido van Rossum (home page: http://www.python.org/~guido/)