[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Paul Moore]

On 22 November 2017 at 16:38, Ivan Levkivskyi <levkivskyi at gmail.com> wrote:
> On 22 November 2017 at 17:16, Paul Moore <p.f.moore at gmail.com> wrote:
>>
>> Docs more importantly than PEP IMO. And are you implying that there's
>> a difference between generator expressions and comprehensions? I
>> thought both were intended to behave as if expanded to a function
>> containing nested for loops? Nothing said in this thread so far (about
>> semantics, as opposed to about current behaviour) implies there's a
>> deliberate difference.
>
>
> I think there may be a difference:
>
> comprehension `g = [(yield i) for i in range(3)]` is defined as this code:
>
>     __result = []
>     __i = None
>     try:
>         for __i in range(3):
>             __result.append(yield __i)
>         g = __result
>     finally:
>         del __result, __i

Not in the docs, it isn't... The docs explicitly state that a new
scope is involved.

People may *intuitively understand it* like this, but it's not the
definition. If it were the definition, I'd be fine, as people can
reason about it and know that the conclusions they come to (no matter
how unintuitive) are accurate.

But if it's how to *understand* what a list comprehension does, then
different rules apply - corner cases may work differently, but
conversely the cases that work differently have to clearly be corner
cases, otherwise it's no use as a mental model.

> while `g = list((yield i) for i in range(3))` is defined as this code:
>
>     def __gen():
>        for i in range(3):
>          yield (yield i)
>     g = list(__gen())

Again, not in the docs.

>  Although these two definitions are equivalent in simple cases (like having
> `f(i)` instead of `yield i`)

And if they don't imply list(...) is the same as [...] then that will
come as a surprise to many people.

> But this is debatable, I think before we move to other points we need to
> agree on the clear definitions of semantics of generator expressions and
> comprehensions.

Absolutely. And I'd like those semantics to be expressed in a way that
doesn't need "except when await/async is involved" exceptions.

Paul