[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Ivan Levkivskyi]

OK, so my 24 hours are over :-)


On 24 November 2017 at 01:50, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 23 November 2017 at 23:04, Ivan Levkivskyi <levkivskyi at gmail.com>
> wrote:
>
>> I don't see why this particular case qualifies for such a radical measure
>> as an exception to syntactic rules,
>> instead of just fixing it (sorry Nick :-)
>>
>
> I've posted in more detail about this to the issue tracker, but the
> argument here is: because making it behave differently from the way it does
> now while still hiding the loop iteration variable potentially requires
> even more radical revisions to the lexical scoping rules :)
>
If somebody can come up with a clever trick to allow yield inside a
> comprehension to jump levels in a relatively intuitive way, that would
> actually be genuinely cool, but the lexical scoping rules mean it's
> trickier than it sounds.
>

"potentially" is the key word here. The plan is to avoid "more radical
revisions".


> Now that I frame the question that way, though, I'm also remembering that
> we didn't have "yield from" yet when I wrote the current comprehension
> implementation, and given that, it may be as simple as having an explicit
> yield expression in a comprehension imply delegation to a subgenerator.
>

My understanding is that this is exactly how async comprehensions are
currently implemented and why they work as one would naively expect, i.e.
`await` is "bound" to the surrounding async def, not to the implicit scope
async def. So that an async comprehension is just equivalent to a for-loop.
However, although "implicit yield from" solution is simpler than the one
proposed by Serhiy, I still more like the latter one. The former has its
strange cases, for example I mentioned before in this thread:

>>> async def f():
...     for i in range(3):
...         yield i
...
>>> async def g():
...     return [(yield i) async for i in f()]
...
>>> g().send(None)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<stdin>", line 2, in g
TypeError: object async_generator can't be used in 'await' expression

My understanding is that the strange error is exactly because of the
implicit `yield from`. With Serhiy's approach this would work.
Another minor bonus of Serhiy's idea is a performance gain: we will not
push an execution frame.

--
Ivan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171124/f81e3135/attachment.html>