[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev]

On 27.06.2018 5:36, Guido van Rossum wrote:
> [This is my one response today]
>
> On Mon, Jun 25, 2018 at 12:40 PM Terry Reedy <tjreedy at udel.edu 
> <mailto:tjreedy at udel.edu>> wrote:
>
>     On 6/24/2018 7:25 PM, Guido van Rossum wrote:
>     > I'd wager that the people who might be most horrified about it
>
>     the (b) scoping rule change
>
>     > would be people who feel strongly that the change to the
>     > comprehension scope rules in Python 3 is a big improvement,
>
>     I might not be one of those 'most horrified' by (b), but I
>     increasingly
>     don't like it, and I was at best -0 on the comprehension scope
>     change.
>     To me, iteration variable assignment in the current scope is a
>     non-problem.  So to me the change was mostly useless churn. Little
>     benefit, little harm.  And not worth fighting when others saw a
>     benefit.
>
>
> Fair enough, and by itself this might not have been enough reason to 
> make the change. But see below.
>
>     However, having made the change to nested scopes, I think we should
>     stick with them.  Or repeal them.  (I believe there is another way to
>     isolate iteration names -- see  below).  To me, (b) amounts to half
>     repealing the nested scope change, making comprehensions half-fowl,
>     half-fish chimeras.
>
>
> That depends on how you see it -- to me (b) just means that there's an 
> implicit nonlocal[1] to make the assignment have the (desirable) 
> side-effect.
>
> The key thing to consider here is whether that side-effect is in fact 
> desirable. For me, the side-effect of the comprehension's loop control 
> variable was never desirable -- it was just an implementation detail 
> leaking out. (And that's different from leaking a regular for-loop's 
> control variable -- since we have 'break' (and 'else') there are some 
> legitimate use cases. But comprehensions try to be expressions, and 
> here the side effect is at best useless and at worst a nasty surprise.)
>
>     > and who are familiar with the difference in implementation
>     > of comprehensions (though not generator expressions) in Python 2
>     vs. 3.
>
>     That I pretty much am, I think.  In Python 2, comprehensions (the
>     fish)
>     were, at least in effect, expanded in-line to a normal for loop.
>     Generator expressions (the fowls) were different.  They were, and
>     still
>     are, expanded into a temporary generator function whose return
>     value is
>     dropped back into the original namespace.  Python 3 turned
>     comprehensions (with 2 news varieties thereof) into fowls also,
>     temporary functions whose return value is dropped back in the
>     original
>     namespace.  The result is that a list comprehension is equivalent to
>     list(generator_ expression), even though, for efficiency, it is not
>     implemented that way.  (To me, this unification is more a benefit
>     than
>     name hiding.)
>
>
> Right, and this consistency convinced me that the change was worth it. 
> I just really like to be able to say "[... for ...]" is equivalent to 
> "list(... for ...)", and similar for set and dict.

"A shorthand to list()/dict()/set()" is actually how I thought of 
comprehensions when I studied them. And I was actually using list() in 
my code for some time before I learned of their existence.

>     (b) proposes to add extra hidden code in and around the temporary
>     function to partly undo the isolation.
>
>
> But it just adds a nonlocal declaration. There's always some hidden 
> code ('def' and 'return' at the very least).
>
>     list comprehensions would no
>     longer be equivalent to list(generator_expression), unless
>     generator_expressions got the same treatment, in which case they
>     would
>     no longer be equivalent to calling the obvious generator function.
>     Breaking either equivalence might break someone's code.
>
>
> Ah, there's the rub! I should probably apologize for not clarifying my 
> terminology more. In the context of PEP 572, when I say 
> "comprehensions" I include generators! PEP 572 states this explicitly 
> (https://github.com/python/peps/blame/master/pep-0572.rst#L201-L202).
>
> Certainly PEP 572 intends to add that implicit nonlocal to both 
> comprehensions and generator expressions. (I just got really tired of 
> writing that phrase over and over, and at some point I forgot that 
> this is only a parenthetical remark added in the PEP's latest 
> revision, and not conventional terminology -- alas. :-)
>
> Part (b) of PEP 572 does several things of things to *retain* consistency:
>
> - The target of := lives in the same scope regardless of whether it 
> occurs in a comprehension, a generator expression, or just in some 
> other expression.
>
> - When it occurs in a comprehension or generator expression, the scope 
> is the same regardless of whether it occurs in the "outermost 
> iterable" or not.
>
> If we didn't have (b) the target would live in the 
> comprehension/genexpr scope if it occurred in a comprehension/genexp 
> but outside its "outermost iterable", and in the surrounding scope 
> otherwise.
>
>     ---
>
>     How loop variables might be isolated without a nested scope: After a
>     comprehension is parsed, so that names become strings, rename the
>     loop
>     variables to something otherwise illegal.  For instance, i could
>     become
>     '<i>', just as lambda becomes '<lambda>' as the name of the resulting
>     function.  Expand the comprehension as in Python 2, except for
>     deleting
>     the loop names along with the temporary result name.
>
>     Assignment expressions within a comprehension would become assignment
>     expressions within the for loop expansion and would automatically
>     add or
>     replace values in the namespace containing the comprehension.  In
>     other
>     words, I am suggesting that if we want name expressions in
>     comprehensions to act as they would in Python 2, then we should
>     consider
>     reverting to an altered version of the Python 2 expansion.
>
>
> Possibly this is based on a misunderstanding of my use of 
> "comprehensions". Also, since your trick can only be used for 
> list/set/dict comprehensions, but not for generator expressions (at 
> least I assume you don't want it there) it would actually *reduce* 
> consistency between list/set/dict comprehensions and generator 
> expressions.
>
>     ---
>
>     In any case, I think (b) should be a separate PEP linked to a PEP for
>     (a).  The decision for (a) could be reject (making (b) moot), accept
>     with (b), or accept unconditionally (but still consider (b)).
>
>
> For me personally, (b) makes the PEP more consistent, so I'm not in 
> favor of breaking up the PEP. But we can certainly break up the 
> discussion -- that's why I started using the labels (a) and (b).
> ----------
> [1] Sometimes it's an implicit global instead of an implicit nonlocal 
> -- when there's already a global for the same variable in the target 
> scope.
>
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru

-- 
Regards,
Ivan

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20180627/7c11a308/attachment-0001.html>