[Python-Dev] Informal educator feedback on PEP 572 (was Re: 2018 Python Language Summit coverage, last part)

[Ivan Pozdeev]

On 27.06.2018 16:49, Steven D'Aprano wrote:
> On Wed, Jun 27, 2018 at 08:00:20AM -0400, Eric V. Smith wrote:
>> On 6/27/2018 7:08 AM, Chris Angelico wrote:
>>> It gets funnier with nested loops. Or scarier. I've lost the ability
>>> to distinguish those two.
>>>
>>> def test():
>>>      spam = 1
>>>      ham = 2
>>>      vars = [key1+key2 for key1 in locals() for key2 in locals()]
>>>      return vars
>>>
>>> Wanna guess what that's gonna return?
>> I'm not singling out Chris here, but these discussions would be easier
>> to follow and more illuminating if the answers to such puzzles were
>> presented when they're posed.
> You can just copy and paste the function into the interactive
> interpreter and run it :-)
>
> But where's the fun in that? The point of the exercise is to learn first
> hand just how complicated it is to try to predict the *current* scope
> behaviour of comprehensions. Without the ability to perform assignment
> inside them, aside from the loop variable, we've managed to avoid
> thinking too much about this until now.
>
> It also demonstrates the unrealisticness of treating comprehensions as a
> separate scope -- they're hybrid scope, with parts of the comprehension
> running in the surrounding local scope, and parts running in an sublocal
> scope.
>
> Earlier in this thread, Nick tried to justify the idea that
> comprehensions run in their own scope, no matter how people think of
> them -- but that's an over-simplification, as Chris' example above
> shows. Parts of the comprehension do in fact behave exactly as the naive
> model would suggest (even if Nick is right that other parts don't).
>
> As complicated and hairy as the above example is, (1) it is a pretty
> weird thing to do, so most of us will almost never need to consider it;
> and (2) backwards compatibility requires that we live with it now (at
> least unless we introduce a __future__ import).
>
> If we can't simplify the scope of comprehensions, we can at least
> simplify the parts that actually matters. What matters are the loop
> variables (already guaranteed to be sublocal and not "leak" out of the
> comprehension) and the behaviour of assignment expressions (open to
> discussion).
>
> Broadly speaking, there are two positions we can take:
>
> 1. Let the current implementation of comprehensions as an implicit
> hidden function drive the functionality; that means we duplicate the
> hairiness of the locals() behaviour seen above, although it won't be
> obvious at first glance.
>
> What this means in practice is that assignments will go to different
> scopes depending on *where* they are in the comprehension:
>
>      [ expr   for x in iter1  for y in iter2  if cond   ...]
>      [ BBBBBB for x in AAAAAA for y in BBBBBB if BBBBBB ...]
>
> Assignments in the section marked "AAAAAA" will be in the local scope;
> assignments in the BBBBBB sections will be in the sublocal scope. That's
> not too bad, up to the point you try to assign to the same name in
> AAAAAA and BBBBBB. And then you are likely to get confusing hard to
> debug UnboundLocalErrors.

This isn't as messy as you make it sound if you remember that the 
outermost iterable is evaluated only once at the start and all the 
others -- each iteration.
Anyone using comprehensions has to know this fact.
The very readable syntax also makes it rather straightforward (though 
admittedly requiring some hand-tracing) to figure out what is evaluated 
after what.

>
> 2. Or we can keep the current behaviour for locals and the loop
> variables, but we can keep assignment expressions simple by ensuring
> they always bind to the enclosing scope. Compared to the complexity of
> the above, we have the relatively straight forward:
>
>      [ AAAAAA for x in AAAAAA for y in AAAAAA if AAAAAA ...]
>
> The loop variables continue to be hidden away in the invisible, implicit
> comprehension function, where they can't leak out, while explicit
> assignments to variables (using := or given or however it is spelled)
> will always go into the surrounding local scope, like they do in every
> other expression.
>
> Does it matter that the implementation of this requires an implicit
> nonlocal declaration for each assignment? No more than it matters that
> comprehensions themselves require an implicit function.
>
> And what we get out of this is simpler semantics at the Python level:
>
> - Unless previous declared global, assignment expressions always bind to
> the current scope, even if they're inside a comprehension;
>
> - and we don't have to deal with the oddity that different bits of a
> comprehension run in different scopes (unless we go out of our way to
> use locals()); merely using assignment expressions will just work
> consistently and simply, and loop variables will still be confined to
> the comprehension as they are now.
>
>

-- 
Regards,
Ivan