[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Chris Angelico]

On Sun, Nov 16, 2014 at 2:21 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 16 November 2014 00:37, Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Sun, Nov 16, 2014 at 1:13 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> >> For certain situations, a simpler and fully backward-compatible
>> >> solution may be sufficient: when a generator returns, instead of
>> >> raising ``StopIteration``, it raises a specific subclass of
>> >> ``StopIteration`` which can then be detected.  If it is not that
>> >> subclass, it is an escaping exception rather than a return statement.
>> >
>> > There's an additional subtlety with this idea: if we add a new
>> > GeneratorReturn exception as a subclass of StopIteration, then generator
>> > iterators would likely also have to change to replace GeneratorReturn
>> > with a
>> > regular StopIteration (chaining appropriately via __cause__, and copying
>> > the
>> > return value across).
>>
>> Would have to do so automatically, meaning this is no simpler than the
>> current proposal? Or would have to be always explicitly written to
>> handle it?
>
> When GeneratorReturn escaped a generator frame, the interpreter would
> automatically convert it into an ordinary StopIteration instance.

Okay, let me see if I have this straight. When a 'return' statement
(including an implicit one at end-of-function) is encountered in any
function which contains a 'yield' statement, it is implemented as
"raise GeneratorReturn(value)" rather than as "raise
StopIteration(value)" which is the current behaviour. However, if any
GeneratorReturn would be raised in any way other than the 'return'
statement, it would magically become a StopIteration instead. Is that
correct?

This does sound simpler. All the magic is in the boundary of the
generator itself, nothing more. If a __next__ method raises either
StopIteration or GeneratorReturn, or if any other function raises
them, there's no special handling.

Question: How does it "become" StopIteration? Is a new instance of
StopIteration formed which copies in the other's ``value``? Is the
type of this exception magically altered? Or is it a brand new
exception with the __cause__ or __context__ set to carry the original?

>> If anyone's
>> still using an old version of contextlib2 once 3.7 comes along, it'll
>> break; but is there any reason to use Python 3.7 with a contextlib
>> from elsewhere than its standard library?
>
>
> Same reason folks use it now: consistent behaviour and features across a
> range of Python versions.
>
> However, that's not the key point - the key point is that working through
> the exact changes that would need to be made in contextlib persuaded me that
> I was wrong when I concluded that contextlib wouldn't be negatively
> affected.
>
> It's not much more complicated, but if we can find a fully supported example
> like that in the standard library, what other things might folks be doing
> with generators that *don't* fall into the category of "overly clever code
> that we don't mind breaking"?

Fair enough. The breakage is a known problem, though; whatever's done
is likely to cause at least some issues. If the alternate you describe
above will break less (or almost none), then it'll be the best option.

>> (I'm not familiar with
>> contextlib2 or what it offers.)
>
> contexlib2 ~= 3.3 era contextlib that runs as far back as 2.6 (I initially
> created it as a proving ground for the idea that eventually become
> contextlib.ExitStack).

Thanks, I figured it'd be like that. Since contextlib exists in 2.7,
is contextlib2 meant to be legacy support only?

ChrisA