[New-bugs-announce] [issue14220] "yield from" kills generator on re-entry

[Stefan Behnel]

New submission from Stefan Behnel <scoder at users.sourceforge.net>:

Based on the existing "test_attempted_yield_from_loop" in Lib/test/test_pep380.py, I wrote this test and I wonder why it does not work:

"""
def test_attempted_reentry():
    """
    >>> for line in test_attempted_reentry(): print(line)
    g1: starting
    Yielded: y1
    g1: about to yield from g2
    g2: starting
    Yielded: y2
    g2: about to yield from g1
    g2: caught ValueError
    Yielded: y3
    g1: after delegating to g2
    Yielded: y4
    """
    trace = []
    def g1():
        trace.append("g1: starting")
        yield "y1"
        trace.append("g1: about to yield from g2")
        yield from g2()
        trace.append("g1: after delegating to g2")
        yield "y4"

    def g2():
        trace.append("g2: starting")
        yield "y2"
        trace.append("g2: about to yield from g1")
        try:
            yield from gi
        except ValueError:
            trace.append("g2: caught ValueError")
        else:
            trace.append("g1 did not raise ValueError on reentry")
        yield "y3"
    gi = g1()
    for y in gi:
        trace.append("Yielded: %s" % (y,))
    return trace
"""

In current CPython, I get this:

"""
Failed example:
    for line in test_attempted_reentry(): print(line)
Expected:
    g1: starting
    Yielded: y1
    g1: about to yield from g2
    g2: starting
    Yielded: y2
    g2: about to yield from g1
    g2: caught ValueError
    Yielded: y3
    g1: after delegating to g2
    Yielded: y4
Got:
    g1: starting
    Yielded: y1
    g1: about to yield from g2
    g2: starting
    Yielded: y2
    g2: about to yield from g1
    g2: caught ValueError
    Yielded: y3
"""

Even though I catch the ValueError (raised on generator reentry) at the position where I run the "yield from", the outer generator (g1) does not continue to run after the termination of g2. It shouldn't normally have an impact on the running g1 that someone attempts to jump back into it, but it clearly does here.

I noticed this while trying to adapt the implementation for Cython, because the original test was one of the few failing cases and it made the code jump through the generator support code quite wildly.

----------
components: Interpreter Core
messages: 155072
nosy: scoder
priority: normal
severity: normal
status: open
title: "yield from" kills generator on re-entry
type: behavior
versions: Python 3.3

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue14220>
_______________________________________