[Python-Dev] PEP 334 - Simple Coroutines via SuspendIteration

[Armin Rigo]

Hi,

I agree with Samuele that the proposal is far too vague currently.  You should
try to describe what precisely should occur in each situation.

A major problem I see with the proposal is that you can describe what should
occur in some situations by presenting source code snippets; such descriptions
correspond easily to possible semantics at the bytecode level.  But bytecode
is not a natural granularity for coroutine issues.  Frames (either of
generators or functions) execute operations that may invoke new frames, and
all frames in the chain except possibly the most recent one need to be
suspended *during* the execution of their current bytecode.  For example, a
generator f() may currently be calling a generator g() with a FOR_ITER
bytecode ('for' statement), a CALL_FUNCTION (calling next()), or actually
anything else like a BINARY_ADD which calls a nb_add implemented in C which
indirectly calls back to Python code.

For this reason it is not reasonably possible to implement restartable
exceptions in general: when an exception is caught, not all the C state is
saved (i.e. you don't know where, *within* the execution of a bytecode, you
should restart).  Your PEP is very similar to restartable exceptions: their
possible semantics are difficult to specify in general.  You may try to do
that to understand what I mean.

This doesn't mean that it is impossible to figure out a more limited concept,
like you are trying to do.  However keeping the "restartable exception" idea
in mind should help focusing on the difficult problems and where restrictions
are needed.

I think that Stackless contains all the solutions in this area, and I'm not
talking about the C stack hacking.  Stackless is sometimes able to switch
coroutines without hacking at the C stack.  I think that if any coroutine
support is ever going to be added to CPython it will be done in a similar way.  
(Generators were also inspired by Stackless, BTW.)  (Also note that although
the generator syntax is nice and helpful, it would have been possible to write
generators without any custom 'yield' syntax if we had restartable exceptions;
this makes the latter idea more general and independent from generators.)


A bientôt,

Armin.