[Python-Dev] Minimal 'stackless' PEP using generators?

[Clark C. Evans]

I just read the thread 'Stackless Python' in June 2004 on python-dev and
was wondering if you'd comment on a simpler cooperative mechanism, via a
small hack to generators:

1.  The PEP would introduce a new 'builtin' class called 'Cooperate'

2.  Generator semantics would be altered so that 'yield X', where X
    is an instance of Cooperate, would automagically propigate to
    the outer-most non-generator.

In for example,

    >>> def MyCooperate(magic.Cooperate):
    >>>     __str__(self): return "cooperate"
    >>>
    >>> def top():
    >>>     yield "one"
    >>>     yield MyCooperate()
    >>>     yield "two"
    >>>
    >>> def middle():
    >>>     """ intermediate generator _only_ sees one and two """
    >>>     for x in top():
    >>>         print "middle", x
    >>>         yield x
    >>>
    >>> def lower():
    >>>     """ this is not a generator, so it sees cooperate """
    >>>     for x in middle():
    >>>         print "lower", x
    >>>
    >>> lower()

    middle one
    lower one
    lower cooperate
    middle two
    lower two

With these two changes, the "lower" function could be an async reactor
like those found in Twisted, or async core.  While the result isn't true
coutines, it would be a huge improvement for those who would like to do
async coding.   I've done something similar with Twisted called Flow [1]
and it works well, with the exception of being a painful syntax hack and
being quite slow.   If this was moved into Python's core, we'd get most
of the advantages of coroutines without the full cost.

Thoughts?

Clark

[1] http://twistedmatrix.com/documents/current/howto/flow

-- 
Clark C. Evans                      Prometheus Research, LLC.
                                    http://www.prometheusresearch.com/
    o                               office: +1.203.777.2550 
  ~/ ,                              mobile: +1.203.444.0557 
 //
((   Prometheus Research: Transforming Data Into Knowledge
 \\  ,
   \/    - Research Exchange Database
   /\    - Survey & Assessment Technologies
   ` \   - Software Tools for Researchers
    ~ *