[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Terry Reedy]

On 6/24/2018 7:25 PM, Guido van Rossum wrote:
> I'd wager that the people who might be most horrified about it

the (b) scoping rule change

> would be people who feel strongly that the change to the
> comprehension scope rules in Python 3 is a big improvement,

I might not be one of those 'most horrified' by (b), but I increasingly 
don't like it, and I was at best -0 on the comprehension scope change. 
To me, iteration variable assignment in the current scope is a 
non-problem.  So to me the change was mostly useless churn.  Little 
benefit, little harm.  And not worth fighting when others saw a benefit.

However, having made the change to nested scopes, I think we should 
stick with them.  Or repeal them.  (I believe there is another way to 
isolate iteration names -- see  below).  To me, (b) amounts to half 
repealing the nested scope change, making comprehensions half-fowl, 
half-fish chimeras.

> and who are familiar with the difference in implementation 
> of comprehensions (though not generator expressions) in Python 2 vs. 3.

That I pretty much am, I think.  In Python 2, comprehensions (the fish) 
were, at least in effect, expanded in-line to a normal for loop. 
Generator expressions (the fowls) were different.  They were, and still 
are, expanded into a temporary generator function whose return value is 
dropped back into the original namespace.  Python 3 turned 
comprehensions (with 2 news varieties thereof) into fowls also, 
temporary functions whose return value is dropped back in the original 
namespace.  The result is that a list comprehension is equivalent to 
list(generator_ expression), even though, for efficiency, it is not 
implemented that way.  (To me, this unification is more a benefit than 
name hiding.)

(b) proposes to add extra hidden code in and around the temporary 
function to partly undo the isolation.  list comprehensions would no 
longer be equivalent to list(generator_expression), unless 
generator_expressions got the same treatment, in which case they would 
no longer be equivalent to calling the obvious generator function. 
Breaking either equivalence might break someone's code.
---

How loop variables might be isolated without a nested scope: After a 
comprehension is parsed, so that names become strings, rename the loop 
variables to something otherwise illegal.  For instance, i could become 
'<i>', just as lambda becomes '<lambda>' as the name of the resulting 
function.  Expand the comprehension as in Python 2, except for deleting 
the loop names along with the temporary result name.

Assignment expressions within a comprehension would become assignment 
expressions within the for loop expansion and would automatically add or 
replace values in the namespace containing the comprehension.  In other 
words, I am suggesting that if we want name expressions in 
comprehensions to act as they would in Python 2, then we should consider 
reverting to an altered version of the Python 2 expansion.
---

In any case, I think (b) should be a separate PEP linked to a PEP for 
(a).  The decision for (a) could be reject (making (b) moot), accept 
with (b), or accept unconditionally (but still consider (b)).

-- 
Terry Jan Reedy