[Python-ideas] Using yield inside a comprehension.

[Chris Angelico]

On Wed, Nov 27, 2013 at 12:54 AM, Jonathan Slenders
<jonathan at slenders.be> wrote:
> Where do I find the PEP that describes that the following statement assigns
> a generator object to `values`?
> values = [ (yield x) for x in range(10) ]
>
> I assume it's equivalent to the following:
> values = (x for x in range(10))
>
>
> The reason for asking this is that I see no point in using the first syntax
> and that the first has another meaning in Python 2.7, if used inside a
> function.

No, it's not the same; after yielding the ten values, the first one
then _returns_ the list. It's a weird and confusing syntax to use for
such a thing, but fundamentally it's (more or less) this:

def listcomp():
    ret = []
    for x in range(10):
        ret.append((yield x))
    return ret
values = listcomp()

Worded like that, you should be able to see what listcomp does. It
yields the integers 0 through 9, collects up any values sent to it (or
None if you use next()), and returns that collection as a list, which
makes it the value of the StopIteration.

If you ignore the value associated with StopIteration, then the two
are equivalent:

>>> values = (x for x in range(10))
>>> list(values)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> values = [ (yield x) for x in range(10) ]
>>> list(values)
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

But not otherwise:

>>> values = (x for x in range(10))
>>> next(values)
0
>>> values.send("asdf")
1
...
>>> values.send("asdf")
9
>>> values.send("asdf")
Traceback (most recent call last):
  File "<pyshell#253>", line 1, in <module>
    values.send("asdf")
StopIteration

>>> values = [ (yield x) for x in range(10) ]
>>> next(values)
0
>>> values.send("asdf")
1
... as above ...
>>> values.send("asdf")
9
>>> values.send("asdf")
Traceback (most recent call last):
  File "<pyshell#265>", line 1, in <module>
    values.send("asdf")
StopIteration: ['asdf', 'asdf', 'asdf', 'asdf', 'asdf', 'asdf',
'asdf', 'asdf', 'asdf', 'asdf']


This is an example of Python doing exactly what it's told, even if you
point the gun at your toes and pull the trigger. It's like creating a
class in which __add__ prints out the value of the object and __repr__
deletes the third element and returns it. Using a list comprehension
to create a generator is technically legal, but outside of the
International Obfuscated Python Code Contest, seldom advisable.

The difference in 2.7 is (if I've understood correctly) to do with the
way a list comp is now part of a function. Others will correct me if
I'm wrong, but I believe the equivalent 2.7 code (what I described
above was the 3.3 code) would look like this (again, hand-waving some
details):

_ = []
for x in range(10):
    _.append((yield x))
values = _

So the yield would take place in the context of the surrounding
function, turning _it_ into a generator - and then assigning to values
the list of ten send()s, or ten Nones.

>>> def foo():
    values = [ (yield x) for x in range(10) ]
    print(values)

>>> list(foo())
[None, None, None, None, None, None, None, None, None, None]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

First line comes from print, second line is what list(foo()) returned.

ChrisA