[Python-Dev] Please reconsider PEP 479.

[Nick Coghlan]

On 28 November 2014 at 02:52, Guido van Rossum <guido at python.org> wrote:
> On Thu, Nov 27, 2014 at 3:04 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> If compatibility with older Python versions is needed, then you could
>> put something like the following in a compatibility module:
>>
>>     try:
>>         from itertools import allow_implicit_stop
>>     except ImportError:
>>         # Allowing implicit stops is the default in older versions
>>         def allow_implicit_stop(g):
>>             return g
>
>
> I understand that @allow_import_stop represents a compromise, an attempt at
> calming the waves that PEP 479 has caused. But I still want to push back
> pretty hard on this idea.
>
> - It means we're forever stuck with two possible semantics for StopIteration
> raised in generators.
>
> - It complicates the implementation, because (presumably) a generator marked
> with @allow_stop_import should not cause a warning when a StopIteration
> bubbles out -- so we actually need another flag to silence the warning.

Ugh, you're right. I'd missed that :(

> - I don't actually know whether other Python implementations have the
> ability to copy code objects to change flags.

I was originally thinking that implicitly catching the RuntimeError
and converting it back to StopIteration could be an acceptable "worst
case" implementation, but I subsequently realised that interacts
differently with yield from than the status quo does.

> - It actually introduces a new incompatibility, that has to be solved in
> every module that wants to use it (as you show above), whereas just putting
> try/except around unguarded next() calls is fully backwards compatible.
>
> - Its existence encourage people to use the decorator in favor of fixing
> their code properly.
>
> - The decorator is so subtle that it probably needs to be explained to
> everyone who encounters it (and wasn't involved in this PEP discussion).
> Because of this I would strongly advise against using it to "fix" the
> itertools examples in the docs; it's just too magical. (IIRC only 2 examples
> actually depend on this.)

Yeah, if not for the status quo, there's no way I'd have suggested it
at all. As it is, you've persuaded me that preserving this capability
indefinitely at the eval loop level isn't worth the extra complexity
(in particular, I'd missed the "add yet another flag to suppress the
warning" issue).

So now I'm wondering if the peephole optimiser could be updated to
pick up the "except -> return" idiom...

> Let me also present another (minor) argument for PEP 479. Sometimes you want
> to take a piece of code presented as a generator and turn it into something
> else. You can usually do this pretty easily by e.g. replacing every "yield"
> by a call to print() or list.append(). But if there are any bare next()
> calls in the code you have to beware of those. If the code was originally
> written without relying on bare next(), the transformation would have been
> easier.

+1

The scenario you describe there strikes me as the statement level
equivalent of the behavioural discrepancies between calling next() in
a generator expression vs doing it in any other kind of comprehension.
In the function definition case, once the "yield" is removed from
elsewhere in the function (so its no longer a generator), it changes
the semantics of any unguarded next() calls.

That's the kind of side effect that's pretty easy for both automated
testing and code review to miss.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia