[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Nick Coghlan]

On 16 November 2014 00:37, Chris Angelico <rosuav at gmail.com> wrote:

> On Sun, Nov 16, 2014 at 1:13 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> >> For certain situations, a simpler and fully backward-compatible
> >> solution may be sufficient: when a generator returns, instead of
> >> raising ``StopIteration``, it raises a specific subclass of
> >> ``StopIteration`` which can then be detected.  If it is not that
> >> subclass, it is an escaping exception rather than a return statement.
> >
> > There's an additional subtlety with this idea: if we add a new
> > GeneratorReturn exception as a subclass of StopIteration, then generator
> > iterators would likely also have to change to replace GeneratorReturn
> with a
> > regular StopIteration (chaining appropriately via __cause__, and copying
> the
> > return value across).
>
> Would have to do so automatically, meaning this is no simpler than the
> current proposal? Or would have to be always explicitly written to
> handle it?
>

When GeneratorReturn escaped a generator frame, the interpreter would
automatically convert it into an ordinary StopIteration instance.

It's still simpler because it won't need the __future__ dance (as it
doesn't involve any backwards incompatible changes).


>  > I definitely see value in adding a GeneratorReturn subclass to be able
> to
> > tell the "returned" vs "raised StopIteration" cases apart from outside
> the
> > generator (the current dance in contextlib only works because we have
> > existing knowledge of the exact exception that was thrown in). I'm
> > substantially less convinced of the benefit of changing generators to no
> > longer suppress StopIteration. Yes, it's currently a rather odd corner
> case,
> > but changing it *will* break code (at the very least, anyone using an old
> > version of contextlib2, or who are otherwise relying on their own copy of
> > contextlib rather than standard library one).
>
> This is why it's proposed to use __future__ to protect it.


Using __future__ still imposes a large cost on the community - docs need
updating, code that relies on the existing behaviour has to be changed,
developers need to adjust their mental models of how the language works.

There needs to be a practical payoff for those costs - and at the moment,
it's looking like we can actually get a reasonably large fraction of the
gain without most of the pain by instead pursuing Guido's idea of a
separate StopIteration subclass to distinguish returning from the outermost
generator frame from raising StopIteration elsewhere in the generator.


> If anyone's
> still using an old version of contextlib2 once 3.7 comes along, it'll
> break; but is there any reason to use Python 3.7 with a contextlib
> from elsewhere than its standard library?


Same reason folks use it now: consistent behaviour and features across a
range of Python versions.

However, that's not the key point - the key point is that working through
the exact changes that would need to be made in contextlib persuaded me
that I was wrong when I concluded that contextlib wouldn't be negatively
affected.

It's not much more complicated, but if we can find a fully supported
example like that in the standard library, what other things might folks be
doing with generators that *don't* fall into the category of "overly clever
code that we don't mind breaking"?

>
> (I'm not familiar with
> contextlib2 or what it offers.)
>

contexlib2 ~= 3.3 era contextlib that runs as far back as 2.6 (I initially
created it as a proving ground for the idea that eventually become
contextlib.ExitStack).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141116/8c31ae21/attachment.html>