[Python-ideas] PEP 479: Change StopIteration handling inside generators

[Guido van Rossum]

(I'm catching up on this thread from the end.)

On Sun, Nov 16, 2014 at 5:29 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Mon, Nov 17, 2014 at 11:03 AM, Steven D'Aprano <steve at pearwood.info>
> wrote:
> > On Mon, Nov 17, 2014 at 11:05:01AM +1300, Greg Ewing wrote:
> >> Chris Angelico wrote:
> >> >if you call on someone else's generator,
> >> >and that someone hasn't applied the __future__ directive, you'll be in
> >> >the current situation of not being able to distinguish 'return' from
> >> >'raise StopIteration'. But for your own generators, you can guarantee
> >> >that they're distinct.
> >>
> >> This suggests that the use of a __future__ directive is not
> >> really appropriate, since it can affect code outside of the
> >> module with the __future__ directive in it.
> >
> > I don't see how that is different from any other __future__ directive.
> > They are all per-module, and if you gain access to an object from
> > another module, it will behave as specified in the module that created
> > it, not the module that imported it. How is this different?
>
> Well, let's see. For feature in sorted(__future__.all_feature_names):
>
> absolute_import: Affects implementation of a keyword
> barry_as_FLUFL: Not entirely sure what this one actually accomplishes. :)
> division: Changes the meaning of one operator.
> generators: Introduces a keyword
> nested_scopes: Alters the compilation of source to byte-code(?)
> print_function: Removes a keyword
> unicode_literals: Alters the type used for literals
> with_statement: Introduces a keyword
>
> Apart from the joke, it seems that every __future__ directive is there
> to affect the compilation, not execution, of its module: that is, once
> a module has been compiled to .pyc, it shouldn't matter whether it
> used __future__ or not. Regardless of unicode_literals, you can create
> bytes literals with b'asdf' and unicode literals with u'asdf'. I'm not
> entirely sure about division (can you call on true-division without
> the future directive?), but in any case, it's all done at compilation
> time, as can be seen interactively:
>
> Python 2.7.3 (default, Mar 13 2014, 11:03:55)
> [GCC 4.7.2] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
> >>> def division1(x,y):
> ...     return x/y, x//y, x%y
> ...
> >>> from __future__ import division
> >>> def division2(x,y):
> ...     return x/y, x//y, x%y
> ...
> >>> import dis
> >>> dis.dis(division1)
>   2           0 LOAD_FAST                0 (x)
>               3 LOAD_FAST                1 (y)
>               6 BINARY_DIVIDE
>               7 LOAD_FAST                0 (x)
>              10 LOAD_FAST                1 (y)
>              13 BINARY_FLOOR_DIVIDE
>              14 LOAD_FAST                0 (x)
>              17 LOAD_FAST                1 (y)
>              20 BINARY_MODULO
>              21 BUILD_TUPLE              3
>              24 RETURN_VALUE
> >>> dis.dis(division2)
>   2           0 LOAD_FAST                0 (x)
>               3 LOAD_FAST                1 (y)
>               6 BINARY_TRUE_DIVIDE
>               7 LOAD_FAST                0 (x)
>              10 LOAD_FAST                1 (y)
>              13 BINARY_FLOOR_DIVIDE
>              14 LOAD_FAST                0 (x)
>              17 LOAD_FAST                1 (y)
>              20 BINARY_MODULO
>              21 BUILD_TUPLE              3
>              24 RETURN_VALUE
>
> So to make this consistent with all other __future__ directives, there
> would need to be some kind of safe way to define this: perhaps an
> attribute on the generator object. Something like this:
>
> >>> def gen(x):
> ...     yield x
> ...     raise StopIteration(42)
> ...
> >>> g=gen(123)
> >>> list(g)
> [123]
> >>> gen.__distinguish_returns__ = True
> >>> g=gen(123)
> >>> list(g)
> Traceback (most recent call last):
>   File "<stdin>", line 1, in <module>
>   File "<stdin>", line 3, in gen
> StopIteration: 42
>
> The attribute on the function would be what affects behaviour; the
> __future__ directive applies that attribute to all generator functions
> in its module (including genexprs). Once the __future__ directive
> becomes automatic, the attribute can and will be dropped - any code
> which interrogates it MUST be prepared to stop interrogating it once
> the feature applies to all modules.
>
> Does that sound reasonable? Should it be added to the PEP?
>

I agree with you and Steven that this is a fine use of __future__. What a
generator does with a StopIteration that is about to bubble out of its
frame is up to that generator. I don't think it needs to be a flag on the
*function* though -- IMO it should be a flag on the code object. (And the
flag should somehow be transferred to the stack frame when the function is
executed, so the right action can be taken when an exception is about to
bubble out of that frame.)

One other small point: let's change the PEP to just propose RuntimeError,
and move the "some other exception" to the "rejected ideas" section.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20141116/51c6d37d/attachment-0001.html>