[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Guido van Rossum]

[This is my one reply in this thread today. I am trying to limit the amount
of time I spend to avoid another overheated escalation.]

On Mon, Jun 25, 2018 at 4:44 AM Nick Coghlan <ncoghlan at gmail.com> wrote:

> Right, the proposed blunt solution to "Should I use 'NAME = EXPR' or
> 'NAME := EXPR'?" bothers me a bit, but it's the implementation
> implications of parent local scoping that I fear will create a
> semantic tar pit we can't get out of later.
>

Others have remarked this too, but it really bother me that you are
focusing so much on the implementation of parent local scoping rather than
on the "intuitive" behavior which is super easy to explain -- especially to
someone who isn't all that familiar (or interested) with the implicit scope
created for the loop control variable(s). According to Steven (who noticed
that this is barely mentioned in most tutorials about comprehensions) that
is most people, however very few of them read python-dev.

It's not that much work for the compiler, since it just needs to do a
little bit of (new) static analysis and then it can generate the bytecode
to manipulate closure(s). The runtime proper doesn't need any new
implementation effort. The fact that sometimes a closure must be introduced
where no explicit initialization exists is irrelevant to the runtime --
this only affects the static analysis, at runtime it's no different than if
the explicit initialization was inside `if 0`.

Unfortunately, I think the key rationale for (b) is that if you
> *don't* do something along those lines, then there's a different
> strange scoping discrepancy that arises between the non-comprehension
> forms of container displays and the comprehension forms:
>
>     (NAME := EXPR,) # Binds a local
>     tuple(NAME := EXPR for __ in range(1)) # Doesn't bind a local
> [...]
> Those scoping inconsistencies aren't *new*, but provoking them
> currently involves either class scopes, or messing about with
> locals().
>

In what sense are they not new? This syntax doesn't exist yet.


> The one virtue that choosing this particular set of discrepancies has
> is that the explanation for why they happen is the same as the
> explanation for how the iteration variable gets hidden from the
> containing scope: because "(EXPR for ....)" et al create an implicitly
> nested scope, and that nested scope behaves the same way as an
> explicitly nested scope as far as name binding and name resolution is
> concerned.
>

Yeah, but most people don't think much about that explanation.

You left out another discrepancy, which is more likely to hit people in the
face: according to your doctrine, := used in the "outermost iterable" would
create a local in the containing scope, since that's where the outermost
iterable is evaluated. So in this example

    a = [x := i+1 for i in range(y := 2)]

the scope of x would be the implicit function (i.e. it wouldn't leak) while
the scope of y would be the same as that of a. (And there's an even more
cryptic example, where the same name is assigned in both places.)

This is another detail of comprehensions that I assume tutorials (rightly,
IMO) gloss over because it's so rarely relevant. But it would make the
explanation of how := works in comprehensions more cumbersome: you'd have
to draw attention to the outermost iterable, otherwise "inline assignment
in comprehensions has the same scope as the comprehension's loop control
variable(s)" would lead one to believe that y's scope above would also be
that of the implicit function.


> Parent local scoping tries to mitigate the surface inconsistency by
> changing how write semantics are defined for implicitly nested scopes,
> but that comes at the cost of making those semantics inconsistent with
> explicitly nested scopes and with the read semantics of implicitly
> nested scopes.
>

Nobody thinks about write semantics though -- it's simply not the right
abstraction to use here, you've introduced it because that's how *you*
think about this.


> The early iterations of PEP 572 tried to duck this whole realm of
> potential semantic inconsistencies by introducing sublocal scoping
> instead, such that the scoping for assignment expression targets would
> be unusual, but they'd be consistently unusual regardless of where
> they appeared, and their quirks would clearly be the result of how
> assignment expressions were defined, rather than only showing up in
> how they interacted with other scoping design decisions made years
> ago.
>

There was also another variant in some iteration or PEP 572, after sublocal
scopes were already eliminated -- a change to comprehensions that would
evaluate the innermost iterable in the implicit function. This would make
the explanation of inline assignment in comprehensions consistent again
(they were always local to the comprehension in that iteration of the PEP),
at the cost of a backward incompatibility that was ultimately withdrawn.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180625/b5e98172/attachment.html>