[issue35756] Using `return value` in a generator function skips the returned value on for-loop iteration

[Steven D'Aprano]

Steven D'Aprano <steve+python at pearwood.info> added the comment:

On Fri, Jan 18, 2019 at 12:31:51AM +0000, bryan.koch wrote:

> Thank you both for the clarifications.  I agree these's no bug in 
> `yield from` however is there a way to reference the return value when 
> a generator with a return is invoked using `for val in gen` i.e. when 
> the generator is invoked without delegation?

I don't believe so, because the for loop machinery catches and consumes 
the StopIteration.

Any change to that behaviour would be new functionality that would 
probably need to go through Python-Ideas first.

[...]
> Essentially I have code that looks like
> `
> for value in generator:
>   do thing with value 
>   yield value 
> ` 
> where I need to do something before yielding the value.
> It would be awesome if invoking a generator above would throw a 
> SyntaxError iff it contained a return and it wasn't invoked through 
> `yield from`.

How is the interpreter supposed to know? (I assume you mean for the 
SytnaxError to be generated at compile-time.) Without doing a 
whole-program analysis, there is no way for the interpreter to compile a 
generator:

    def gen():
        yield 1
        return 1

and know that no other piece of code in some other module will never 
call it via a for loop.

> The below isn't valid Python and I'm not sure that it should be but 
> it's what I need to do.
> 
> `
> return_value = for value in generator:
>   do thing with value
>   yield value
> 
> if return_value:
>   do something with return_value
> `

Let me be concrete here. You have a generator which produces a sequence 
of values [spam, eggs, cheese, aardvark] and you need to treat the 
final value, aardvark, differently from the rest:

    do thing with spam, eggs, cheese
    do a different thing with aardvark (if aardvark is a True value)

Am I correct so far?

Consequently you writing this as:

def gen():
    yield spam
    yield eggs
    yield cheese
    return aardvark

Correct?

That's an interesting use-case, but I don't think there is any obvious 
way to solve that right now. Starting in Python 3.8, I think you should 
be able to write:

for x in (final := (yield from gen())):
    do something with x  # spam, eggs, cheese
if final:
    do something different with final  # aardvark

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue35756>
_______________________________________