[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Guido van Rossum]

On Sat, Nov 25, 2017 at 3:24 PM, Greg Ewing <greg.ewing at canterbury.ac.nz>
wrote:

> Nick Coghlan wrote:
>
>     def example():
>>         comp1 = yield from [(yield x) for x in ('1st', '2nd')]
>>         comp2 = yield from [(yield x) for x in ('3rd', '4th')]
>>         return comp1, comp2
>>
>
> If the implicit "yield from" idea seems too magical, then the other
>> direction we could go is to make the immediate "yield from" mandatory,
>>
>
> Whether it's "too magical" or not depends on your stance with
> regard to the implicit function scope of a comprehension. There
> seem to be two irreconcilable schools of thought on that:
>
> (1) It's an implementation detail that happens to be used to
> stop the loop variable from leaking. Most of the time you can
> ignore it and use the nested-loop-expansion mental model.
> Most people seem to think this way in practice.
>
> (2) It's an important part of the semantics of comprehensions
> that needs to be taken into account at all times. This seems
> to be Guido's position.
>
> If you're in school (1), then the current behaviour of yield
> in a comprehension is the thing that's weird and magical.
> Moreover, it's bad magic, because it does something you
> almost certainly don't want. Adding an implicit yield-from
> would make things *less* magical and more understandable.
>
> However, it will break things for people in school (2), who
> are aware of the arcane details and know enough to put the
> yield-from in themselves when needed.
>
> Just because it breaks their code doesn't necessarily mean
> they will find it more magical, though. Generators are
> already sufficiently magical that they're probably smart
> enough to figure out what's going on.
>
> Personally I'm somewhat uncomfortable with having a rule
> that "yield" in a comprehension creates a subgenerator,
> because the syntactic clues that a given "yield" is in a
> comprehension are fairly subtle.
>
> For the same reason, I'm uncomfortable with the nested
> function scope of a comprehension being an official part
> of the semantics. All other function scopes are introduced
> by a very clear piece of syntax, but with comprehensions
> you have to notice a combination of things that don't
> have anything to do with function scopes by themselves.
>
> So I think I've just talked myself into the opinion that
> anything that would allow you to tell whether comprehensions
> have an implicit function scope or not should be banned.
>

That's not an unreasonable position, and one that would leave a door open
to a general design for inner scopes that are not implemented using
functions or code objects (something to ponder for 3.8 perhaps?). In terms
of the current thread the only thing it would disallow seems to be the
current behavior of yield in comprehensions (and perhaps genexprs?).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171125/b76d8df9/attachment-0001.html>