[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Chris Angelico]

On Thu, Nov 20, 2014 at 3:03 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> OTOH, I'm also not sure the status quo is sufficiently problematic to be
> worth changing. Yes, it's a little weird, but is it *that* much weirder than
> the unavoidable issues with exceptions thrown in __next__, __getitem__,
> __getattr__ and other special methods where a particular type of exception
> is handled directly by the interpreter?

If you write __next__, you write in a "raise StopIteration" when it's
done. If you write __getattr__, you write in "raise AttributeError" if
the attribute shouldn't exist. Those are sufficiently explicit that it
should be reasonably clear that the exception is the key. But when you
write a generator, you don't explicitly raise:

def gen():
    yield 1
    yield 2
    yield 3
    return 4

The distinction in __next__ is between returning something and raising
something. The distinction in a generator is between "yield" and
"return". Why should a generator author have to be concerned about one
particular exception having magical meaning?

Imagine this scenario:

def producer():
    """Return user input, or raise KeyboardInterrupt"""
    return input("Enter the next string: ")

def consumer():
    """Process the user's input"""
    while True:
        try:
            command = producer()
        except KeyboardInterrupt:
            break
        dispatch(command)


Okay, now let's make a mock producer:

strings = ["do stuff","do more stuff","blah blah"]
def mock_producer()
    if strings: return strings.pop(0)
    raise KeyboardInterrupt

That's how __next__ works, only with a different exception, and I
think people would agree that this is NOT a good use of
KeyboardInterrupt. If you put a few extra layers in between the
producer and consumer, you'd be extremely surprised that an unexpected
KeyboardInterrupt just quietly terminated a loop. Yet this is exactly
what the generator-and-for-loop model creates: a situation in which
StopIteration, despite not being seen at either end of the code, now
has magical properties. Without the generator, *only* __next__ has
this effect, and that's exactly where it's documented to be.

Does that make for more justification? Unexpected exceptions bubbling
up is better than unexpected exceptions quietly terminating loops?

ChrisA