[Python-Dev] Generator objects and list comprehensions?

[Chris Angelico]

On Wed, Jan 25, 2017 at 4:38 PM, Craig Rodrigues <rodrigc at freebsd.org> wrote:
>
> Glyph pointed this out to me here:
> http://twistedmatrix.com/pipermail/twisted-python/2017-January/031106.html
>
> If I do this on Python 3.6:
>
>>>  [(yield 1) for x in range(10)]
> <generator object <listcomp> at 0x10cd210f8>
>
> If I understand this:
> https://docs.python.org/3/reference/expressions.html#list-displays
> then this is a list display and should give a list, not a generator object.
> Is there a bug in Python, or does the documentation need to be updated?

That looks like an odd interaction between yield expressions and list
comps. Since a list comprehension is actually implemented as a nested
function, your code actually looks more-or-less like this:

>>> def <listcomp>(iter):
    result = []
    for x in iter:
        result.append((yield 1))
    return result
>>> <listcomp>(iter(range(10))
<generator object <listcomp> at 0x10cd210f8>

This function is a generator, and calling it returns what you see
above. If you step that iterator, it'll yield 1 ten times, and then
raise StopIteration with the resulting list.

Based on a cursory examination of the issue at hand, I think what
you're working with might be functioning as a coroutine? If so, you
may find that using "await" instead of "yield" dodges the problem, as
it won't turn the list comp into a generator. But I can't be 100%
certain of that. (Also, that would definitely stop you from having
single-codebase 2.7/3.x code.)

ChrisA