[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Chris Angelico]

On Sat, Nov 22, 2014 at 3:30 AM, Steven D'Aprano <steve at pearwood.info> wrote:
> On Fri, Nov 21, 2014 at 10:50:52PM +1100, Chris Angelico wrote:
>> Yes, this would be affected. This proposal causes a separation of
>> generators and iterators, so it's no longer possible to pretend that
>> they're the same thing.
>
> But generators and iterators *are the same thing*. (Generator functions
> are not iterators, but generators themselves are.) Iterators don't have
> a specific type, but they obey the iterator protocol:

I can write many other factory functions which return iterators. They
are not, themselves, iterators, and therefore should not be expected
to follow iterator protocol.

def gen():
    return iter([1,2])


> py> it = gen()
> py> iter(it) is it
> True
> py> hasattr(it, '__next__')
> True
>
> `it` is an iterator.

The above function works with those tests, too. Generator functions
are functions that return iterators, and the __next__ method of the
returned object is what follows iterator protocol.

>> The main point is one of exceptions being silently suppressed.
>> Iterator protocol involves the StopIteration exception; generator
>> protocol doesn't, yet currently a generator that raises StopIteration
>> will quietly terminate. It's as if every generator is wrapped inside
>> "try: ..... except StopIteration: pass". Would you accept any function
>> being written with that kind of implicit suppression of any other
>> exception?
>
> Yes.
>
> That's how the classic pre-iterator iteration protocol works:
>
> py> class K:
> ...     def __getitem__(self, i):
> ...             if i == 5: raise IndexError
> ...             return i
> ...
> py> x = K()
> py> list(x)
> [0, 1, 2, 3, 4]

That's following getitem protocol, and it's part of that protocol for
the raising of IndexError to be the way of not returning any value.
But what's more surprising is that raising StopIteration will also
silently halt iteration, which I think is not good:

>>> class K:
    def __getitem__(self, i):
        if i == 5: raise StopIteration
        return i

>>> list(K())
[0, 1, 2, 3, 4]

> Context managers support suppressing any exception which occurs:
>
>     If the suite was exited due to an exception, and the return
>     value from the __exit__() method was false, the exception is
>     reraised. If the return value was true, the exception is
>     suppressed, and execution continues with the statement
>     following the with statement.
>
> https://docs.python.org/3/reference/compound_stmts.html#the-with-statement

Context managers get a chance to function like a try/except block. If
one silently and unexpectedly suppresses an exception, it's going to
be surprising; but more likely, it's as clear and explicit as an
actual try/except block. This isn't "as soon as you use a 'with'
block, any XyzError will jump to the end of the block and keep going".

ChrisA