[Python-Dev] Tricky way of of creating a generator via a comprehension expression

[Guido van Rossum]

On Thu, Nov 23, 2017 at 9:06 AM, Serhiy Storchaka <storchaka at gmail.com>
wrote:

> 23.11.17 18:08, Guido van Rossum пише:
>
>> This thread is still going over the speed limit. Don't commit anything
>> without my explicit approval.
>>
>
> I'm not going to write a single line of code while the decision about this
> issue is not made. This is not easy issue.
>

OK, I think Ivan at some point said he was waiting how your fix would look.


> A problem with dropping the "function-ness" of the comprehension by
>> renaming the variables (as Ivan/Serhiy's plan seems to be?) would be what
>> does it look like in the debugger -- can I still step through the loop and
>> print the values of expressions? Or do I have to know the "renamed" names?
>>
>
> Does the debugger supports stepping inside a comprehension? Isn't the
> whole comprehension a single logical line? It this is supported the the
> debugger should see a local variable with a strange name ".0". This is a
> leaked implementation detail. It wouldn't be surprised if details of other
> implementation will be leaked too in the debugger. The debugger can
> unmangle the inner variable names and hide local variables in the outer
> scope.
>

The debugger does stop at each iteration. It does see a local named ".0"
(which of course cannot be printed directly, only using vars()). I suppose
there currently is no way for the debugger to map the variable names to
what they are named in the source, right? (If you think this is a useful
thing to add, please open a separate issue.)


> And my mind boggles when considering a generator expression containing
>> yield that is returned from a function. I tried this and cannot say I
>> expected the outcome:
>>
>>    def f():
>>        return ((yield i) for i in range(3))
>>    print(list(f()))
>>
>> In both Python 2 and Python 3 this prints
>>
>>    [0, None, 1, None, 2, None]
>>
>> Even if there's a totally logical explanation for that, I still don't
>> like it, and I think yield in a comprehension should be banned. From this
>> it follows that we should also simply ban yield  from comprehensions.
>>
>
> This behavior doesn't look correct to me and Ivan. Ivan explained that
> this function should be rough equivalent to
>
>    def f():
>        t = [(yield i) for i in range(3)]
>        return (x for x in t)
>
> which should be equivalent to
>
>    def f():
>        t = []
>        for i in range(3):
>           t.append((yield i))
>        return (x for x in t)
>
> and list(f()) should be [1, 2, 3].
>
> But while I know how to fix yield in comprehensions (just inline the code
> with some additions), I have no ideas how to fix yield in generators. Me
> and Ivan agreed that SyntaxError in this case is better than the current
> behavior. Maybe we will find how to implement the expected behavior.
>

I doubt that anyone who actually wrote that had a valid expectation for it
-- most likely they were coding by random modification or it was the result
of a lack of understanding of generator expressions in general. I think the
syntax error is the best option here.

I also think we should all stop posting for 24 hours as tempers have gotten
heated.

Happy Thanksgiving!

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20171123/af059011/attachment.html>