[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Devin Jeanpierre]

On Sat, Nov 15, 2014 at 1:29 AM, Chris Angelico <rosuav at gmail.com> wrote:
> The interaction of generators and ``StopIteration`` is currently
> somewhat surprising, and can conceal obscure bugs.  An unexpected
> exception should not result in subtly altered behaviour, but should
> cause a noisy and easily-debugged traceback.  Currently,
> ``StopIteration`` can be absorbed by the generator construct.

Specifically it's absorbed by the caller of the generator, because the
caller doesn't know the difference between next(x) raising
StopIteration because the iterator specifically wants to stop, vs
because of accident.

As another alternative, how about a new iterator protocol that is
defined without this ambiguity? Code at the bottom of my post to help
explain:  define a new method __nextx__ which doesn't use
StopIteration for any signalling, instead, it returns None if there
are no values to return, and returns a special value Some(v) if it
wants to return a value v.  Both next(it) and nextx(it) are made to
work for any iterator that is defined using either protocol, but for
loops and Python builtins all use nextx internally. Generators define
__next__ unless you from __future__ import iterators, in which case
they define __nextx__ instead.

In this way, old code can't tell the difference between accidental
StopIteration and deliberate StopIteration, but new code (using nextx
instead of next, and using __future__ import'd generators) can. No
backwards incompatibility is introduced, and you can still insert
StopIteration into a generator and get it back out -- using both
next() where it is ambiguous and nextx() where it is not.

Yes, it's ugly to have two different iterator protocols, but not that
ugly. In fact, this would be Python's third (I have omitted that third
protocol in the below example, for the sake of clarity). I find the
proposed solution more scary, in that it's sort of a "hack" to get
around an old mistake, rather than a correction to that mistake, and
it introduces complexity that can't be removed in principle. (Also,
it's very unusual.)

class Some:
    def __init__(self, value):
        self.value = value

def next(it):
    v = nextx(it)
    if v is None:
        raise StopIteration
    return v.value

def nextx(it):
    if hasattr(it, '__nextx__'):
        v = it.__nextx__()
        if v is None or isinstance(v, Some):
            return v
        raise TypeError("__nextx__ must return Some(...) or None, not
%r" % (v,))
    if hasattr(it, '__next__'):
        try:
            return Some(it.__next__())
        except StopIteration:
            return None
    raise TypeError

-- Devin