question about generators

[Tim Peters]

[Andrew Koenig]
> ...
> Yield actually does one of two very different things depending
> on context.

I expect that's been cleared up by now, so I'll skip a repetition.

> ...
> These facts mean that yield statements break a form of abstraction
> that is commonplace in many other contexts: the ability to take a
> collection of statements and put them in a function.  For example:
>
>         def f():
>                 <statement 1>
>                 <statement 2>
>                 <statement 3>
>                 <statement 4>
>
> Under ordinary circumstances, I can rewrite this as
>
>         def f():
>                 <statement 1>
>                 g()
>                 <statement 4>
>
>         def g():
>                 <statement 2>
>                 <statement 3>
>
> without changing the meaning of the program (provided that statement 2
> and statement 3 do not refer to local variables of f).

More relevant I think is that 2 and 3 must not affect control flow in ways
that change as a result of the move.  For example, if 2 is "return", the
"before" and "after" spellings say quite different things.

> However, if I use yield statements,

Among many other control-flow statements, yes <wink>.

> the abstraction breaks down:

>         def f():
>                 yield 1
>                 yield 2
>                 yield 3
>                 yield 4
>
> is not equivalent to
>
>         def f():
>                 yield 1
>                 g()
>                 yield 4
>
>         def g():
>                 yield 2
>                 yield 3
>
> I think that Tim's "yield every g()" notion is really a way of saying
> ``I want to call g, but I want every yield in g and anything it calls
> to be considered type 2, not type 1.''

I don't buy the type 1-vs-2 distinction (all yields are "type 2" in Python);

    yield every x

is intended as pure sugar for

    for _temp_var in x:
        yield _temp_var

A

    yield every g()

in your "after" f() would make it quite like the "before" spelling.

If you haven't played with the Icon language, you should!  It's a lot of
fun, and it's very educational to program in a language where generators are
ubiquitous (all expressions in Icon are generators, although many of them--
like, say, the integer literal 42 --yield a sequence with only one value).
What I'm calling "yield every" here is closest to what's called "suspend" in
Icon; Icon also has an "every" control structure, which forces its
expression argument to generate all its results, but without any notion of
returning them *to* "a caller".  So, e.g.,

    every f((1 to 3) + (5 to 10))

calls the function f() 18 times, once each for the 18 sums 1+5, 1+6, ...,
3+10.  OK, I'm lying, it may do a lot more than that, if "f" is an
expression that happens to generate multiple callables too.  Like

    every (f|g|h)(42)

calls f(42) then g(42) then h(42).

    suspend (f|g|h)(42)

does the same, but also yields each function-call result to the current
caller.

Generators weren't meant to be central to Python programming, though, and it
took a lot of discipline to keep the Simple Generators PEP simple <0.7
wink>.