[Python-ideas] Yield-From: Finalization guarantees

[Jacob Holm]

Nick Coghlan wrote:
> 4, 3, 2, 1 is the position I've come around to. 
>
> [...snip...]
>
> By adopting position 4, I believe the guarantees for the exception
> handling in the new expression become as simple as possible:
>  - if the subiterator does not provide a throw() method, or the
> exception thrown in is GeneratorExit, then the subiterator's close()
> method (if any) is called and the thrown in exception raised in the
> current frame
>  - otherwise, the exception (including traceback) is passed down to the
> subiterator's throw() method
>   

Below I have attached a heavily annotated version of the expansion that 
I expect for #4.  This version fixes an issue I have forgotten to 
mention where the subiterator is not closed due to an AttributeError 
caused by a missing send method.

> With these semantics, subiterators will be finalised promptly when the
> outermost generator is finalised without any special effort on the
> developer's part and it won't be trivially easy to accidentally suppress
> GeneratorExit.
>   

The way I see it, it will actually be hard to do even on purpose, unless 
you are willing to take a significant performance hit by using a 
non-generator wrapper for every generator.

> To my mind, the practical benefits of such an approach are enough to
> justify the deviation from the general 'inline behaviour' guideline.
>
>   

I disagree, but it seems like I am the only one here that does.   It 
will eliminate a potential pitfall, but will also remove some behavior 
that could have been useful, such as the ability to suppress the 
GeneratorExit if you know what you are doing.

- Jacob


------------------------------------------------------------------------

_i = iter(EXPR)  # Raises TypeError if not an iterable.
try:
    _x = None   # No current exception.
    _y = _i.__next__() # Guaranteed to be there by iter().
    while 1:
        try:
            _s = yield _y
        except BaseException as _e:
            # An exception was thrown in, either by a call to throw() on the generator or implicitly by a call
            # to close().
            _x = _e # Save the thrown-in exception as current.
            if isinstance(_x, GeneratorExit): 
                _m = None  # Don't forward GeneratorExit.
            else:
                _m = getattr(_i, 'throw', None) # Forward any other exception if there is a throw() method.
            if _m is None:
                # Not forwarding. Exit loop and go to finally clause (possibly via "except StopIteration"),
                # which will close _i before reraising _x.
                raise
            _y = _m(_x)
        else:
            if _s is None:
                # Either a send(None) or a __next__(), forward as __next__().
                _x = None # No current exception
                _y = _i.__next__() # Guaranteed to be there by iter().
            else:
                # A send(non-None).  We need to handle the case where the subiterator has no send() method.
                try:
                    _m = _i.send
                except AttributeError as _e:
                    # No send method.  Ensure that the subiterator is closed, then reraise the AttributeError.
                    _x = _e   # Save the AttributeError as the current exception.
                    _m = None # Clear _m so we know _x has not been forwarded.
                    raise     # Exit loop and go to finally clause, which will close _i before reraising _x.
                else:
                    _x = None # No current exception.
                    _y = _m(s)
except StopIteration as _e:
    if _e is _x:
        # If _e was just thrown in, reraise it.  If the exception has been forwarded to the subiterator,
        # the subiterator is assumed closed.  In that case _m will be non-None, so the subiterator will not be
        # closed again by the finally clause.  Conversely, if the exception was not forwarded _m will be None
        # and the finally clause takes care of closing it before reraising the exception.
        raise
    # Normal return.  If we get here, the StopIteration was raised by a __next__(), send() or throw() on the
    # subiterator which will therefore already be closed.  In this case either _x is None or _m is not None, so
    # the the subiterator will not be closed again by the finally clause.
    RESULT = _e.value
finally:
    if _x is not None and _m is None:
        # An exception is active and was not raised by the subiterator.  Explicitly call close before the 
        # exception is automatically reraised by the finally clause.  If close raises an exception, that will
        # take over.
        _m = getattr(_i, 'close', None)
        if _m is not None:
            _m()