[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Chris Angelico]

On Wed, Nov 26, 2014 at 4:30 AM, Wolfgang Maier
<wolfgang.maier at biologie.uni-freiburg.de> wrote:
> Well, I'm not familiar with every implementation detail of the interpreter
> so I can't judge how difficult to implement certain things would be, but one
> solution that I could think of is:

I don't know much about the internal details of CPython either, but
let's just ignore that for the moment and consider specs for the
Python language. AFAIK, not one of the concerns raised (by PEP 479 or
your proposal here) is CPython-specific.

> allow StopIteration to be raised anywhere, but let it bubble up only *one*
> frame.
> So if the next outer frame does not deal with it, the exception would be
> converted to UnhandledStopIteration (or something else) when it's about to
> bubble out of that outer frame.
> The builtin next() would simply reset the frame count by catching and
> reraising StopIteration raised inside its argument (whether that's an
> iterator's __next__ or a generator; note that in this scenario using raise
> StopIteration instead of return inside a generator would remain possible).

Interesting. Makes a measure of sense, and doesn't have much magic to it.

> So different from the current PEP where a StopIteration must be dealt with
> explicitly using try/except only inside generators, but bubbles up
> everywhere else, here StopIteration will be special everywhere, i.e., it
> must be passed upwards explicitly through all frames or will get converted.
>
> Back to Steven's generator expression vs comprehension example:
>
> iterable = [iter([])]
> list(next(x) for x in iterable)
>
> would raise UnhandledStopIteration since there is no way, inside the
> generator expression to catch the StopIteration raised by next(x).

Downside of this is that it's harder to consciously chain iterators,
but maybe that's a cost that has to be paid.

Suggestion for this: Have a new way of "raise-and-return". It's mostly
like raise, except that (a) it can't be caught by a try/except block
in the current function (because there's no point), and (b) it
bypasses the "this exception must not pass unnoticed". It could then
also be used for anything else that needs the "return any object, or
signal lack of return value" option, covering AttributeError and so
on.

So it'd be something like this:

class X:
    def __iter__(self): return self
    def __next__(self):
        if condition: return value
        signal StopIteration

The 'signal' statement would promptly terminate the function (not sure
exactly how it'd interact with context managers and try/finally, but
something would be worked out), and then raise StopIteration in the
calling function. Any other StopIteration which passes out of a
function would become a RuntimeError.

Magic required: Some way of knowing which exceptions should be covered
by this ban on bubbling; also, preferably, some way to raise
StopIteration in the calling function, without losing the end of the
backtrace.

This could be a viable proposal. It'd be rather more complicated than
PEP 479, though, and would require a minimum of five hundred
bikeshedding posts before it comes to any kind of conclusion, but if
you feel this issue is worth it, I'd certainly be an interested
participant in the discussion.

ChrisA