[Python-Dev] comments on PEP 219

[Tim Peters]

[Jeremy Hylton]
> ...
> One other set of issues, that is sort-of out of bounds for this
> particular PEP, is what control features do we want that can only be
> implemented with stackless.  Can we implement generators or coroutines
> efficiently without a stackless approach?

Icon/CLU-style generator/iterators always return/suspend directly to their
immediate caller/resumer, so it's impossible to get a C stack frame "stuck in
the middle":  whenever they're ready to yield (suspend or return), there's
never anything between them and the context that gave them control  (and
whether the context was coded in C or Python -- generators don't care).

While Icon/CLU do not do so, a generator/iterator in this sense can be a
self-contained object, passed around and resumed by anyone who feels like it;
this kind of object is little more than a single Python execution frame,
popped from the Python stack upon suspension and pushed back on upon
resumption.  For this reason, recursive interpreter calls don't bother it:
whenever it stops or pauses, it's at the tip of the current thread of
control, and returns control to "the next" frame, just like a vanilla
function return.  So if the stack is a linear list in the absence of
generators, it remains so in their presence.  It also follows that it's fine
to resume a generator by making a recursive call into the interpreter (the
resumption sequence differs from a function call in that it must set up the
guts of the eval loop from the state saved in the generator's execution
frame, rather than create a new execution frame).

But Guido usually has in mind a much fancier form of generator (note:  contra
PEP 219, I didn't write generator.py -- Guido wrote that after hearing me say
"generator" and falling for Majewski's hypergeneralization of the concept
<0.8 wink>), which can suspend to *any* routine "up the chain".  Then C stack
frames can certainly get stuck in the middle, and so that style of generator
is much harder to implement given the way the interpreter currently works.
In Icon *this* style of "generator" is almost never used, in part because it
requires using Icon's optional "co-expression" facilities (which are optional
because they require hairy platform-dependent assembler to trick the platform
C into supporting multiple stacks; Icon's generators don't need any of that).
CLU has nothing like it.

Ditto for coroutines.