[Python-Dev] PEP 380 (yield from a subgenerator) comments

[Guido van Rossum]

On Tue, Mar 24, 2009 at 4:02 PM, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Guido van Rossum wrote:
>> I really don't like to have things whose semantics is
>> defined in terms of code inlining -- even if you don't mean that as
>> the formal semantics but just as a mnemonic hint.
>
> Think about it the other way around, then. Take any chunk
> of code containing a yield, factor it out into a separate
> function (using whatever techniques you would normally
> use when performing such a refactoring to deal with
> references to variables in the surrounding scope) and
> call it using yield-from. The result should be the same
> as the original unfactored code.

The way I think of it, that refactoring has nothing to do with
yield-from. It's not just variable references -- I used "scope" as a
shorthand for everything that can be done within a function body,
including control flow: try/except/finally,
continue/break/raise/return.

> That's the fundamental reason behind all of this -- to
> make such refactorings possible in a straightforward way.

Well, it solves one particular detail.

>> What happened to the first-order approximation
>>
>>  "yield from X" means roughly the same as "for _x in X: yield x"
>
> Everybody's reaction to that when it's been suggested
> before has been "that's trivial, why bother?" So I've
> been trying to present it in a way that doesn't make it
> appear so trivial.

Maybe you're confusing motivation with explanation? That feedback
seems to tell me that the *motivation* needs more work; but IMO the
*explanation* should start with this simple model and then expand upon
the edge cases.

> Also, my understanding is that a PEP is not meant to be
> a tutorial for naive users, but a document for communicating
> ideas between core Python developers, who are presumably
> savvy enough not to need such watered-down material.

Not quite. PEPs aren't *just* for core developers -- they are also for
communicating to (savvy) developers outside the core group. A good PEP
needs to summarize both the motivation and specification concisely so
prospective readers can quickly determine what it is about, and
whether they care.

> But I'll be happy to add a paragraph about this at the
> beginning if you think it would help.
>
>> But the exact
>> semantics in the light of try/except or try/finally blocks on the
>> stack are incredibly (perhaps impossibly) tricky to get right -- and
>> it probably doesn't matter all that much what exactly happens as long
>> as it's specified in sufficient detail that different implementations
>> behave the same way (apart from obvious GC differences, alas).
>
> This is part of the reason I've been emphasising the
> inlining principle. When pondering what should happen
> in such cases, I've been able to think to myself
> "What would happen if the subgenerator were inlined?"
> Most of the time that makes the answer fairly obvious,
> at least in the case where the subiterator is another
> generator. Then it's a matter of generalising it to
> other iterators.

This is a good way of thinking about use cases, because it helps
deciding how the edge cases should be specified that the simplest
model (my one-liner above) doesn't answer in a useful way. But it
should not be confused with an explanation of the semantics.

-- 
--Guido van Rossum (home page: http://www.python.org/~guido/)