[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Guido van Rossum]

On Mon, Nov 17, 2014 at 4:56 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 17 November 2014 13:34, Chris Angelico <rosuav at gmail.com> wrote:
>
>> Right. So, are there any __future__ directives that have any effect on
>> byte-code? I'm not seeing any, though that may just mean I didn't
>> recognize one.
>>
>
> True division (in Python 2) is a nice simple one to look at, since it just
> swaps one bytecode for another (BINARY_DIVIDE -> BINARY_TRUE_DIVIDE)
>
> >>> def floor_div(x, y):
> ...     return x / y
> ...
> >>> from __future__ import division
> >>> def true_div(x, y):
> ...     return x / y
> ...
> >>> import dis
> >>> dis.dis(floor_div)
>   2           0 LOAD_FAST                0 (x)
>               3 LOAD_FAST                1 (y)
>               6 BINARY_DIVIDE
>               7 RETURN_VALUE
> >>> dis.dis(true_div)
>   2           0 LOAD_FAST                0 (x)
>               3 LOAD_FAST                1 (y)
>               6 BINARY_TRUE_DIVIDE
>               7 RETURN_VALUE
>
> The compiler actually stores a whole pile of useful info on code objects
> that doesn't show up in the disassembly output (switching to Python 3 for
> more up to date dis module goodness):
>
> >>> import dis
> >>> def true_div(x, y):
> ...     return x / y
> ...
> >>> dis.show_code(true_div)
> Name:              true_div
> Filename:          <stdin>
> Argument count:    2
> Kw-only arguments: 0
> Number of locals:  2
> Stack size:        2
> Flags:             OPTIMIZED, NEWLOCALS, NOFREE
> Constants:
>    0: None
> Variable names:
>    0: x
>    1: y
>
> So conveying to the generator iterator whether or not "from __future__
> import generator_return" was in effect would just be a matter of the
> compiler setting a new flag on the generator code object. For *affected
> generators* (i.e. those defined in a module where the new future statement
> was in effect), StopIteration escaping would be considered a RuntimeError.
>
> For almost all code, such RuntimeErrors would look like any other
> RuntimError raised by a broken generator implementation.
>
> The only code which would *have* to change immediately as a "Porting to
> Python 3.5" requirement is code like that in contextlib, which throws
> StopIteration into generators, and currently expects to get it back out
> unmodified. Such code will need to be updated to also handle RuntimError
> instances where the direct cause is the StopIteration exception that was
> thrown in.
>
> Other affected code (such as the "next() bubbling up" groupby example)
> would keep working unless the __future__ statement was in effect.
>

Thanks, this is exactly what I was thinking of. The new flag could be named
REPLACE_STOPITERATION. Then the __future__ import could be named
replace_stopiteration_in_generators (it needs more description than the
flag name because the flag is already defined in the context of a
generator, while the __future__ import must still establish that context).

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141117/7b43a54f/attachment.html>