PEP 255: Simple Generators

[Glyph Lefkowitz]

My $0.02 on the whole issue:

Generators a-la PEP 255 seem like a *very* bad idea to me.

Generators can be implemented through continuations (as shown in
Stackless) or through microthreads (as shown in the various previous
implementations of generators which use *real* threads cooperatively).

Neither microthreads nor continuations are implementable through
generators.  I believe that generators (as described in PEP 255) are a
merely a convenience, but real cooperative multithreading could be very
useful and we would get generators *for free*.

While I am heavily in support of stackless being integrated for many
reasons, I can understand the unfavorable sentiments that some harbor
towards continuations.  They can be nastily abused.  They are
confusing.  They interact very poorly with operator overloading and
exception handling.

However, yeild adds a way of altering control flow without providing a
direct mechanism for accessing that alteration; only using one particular
feature of that alteration.  One real example -- the reason I'm writing
this reponse -- I have written an asynchronous networking library.  
Currently, the API is that the client "protocol" code handles each chunk
of data, maintaining a buffer itself, and then handling an event when a
full message chunk of data is received.  Certain protocols would be much
easier to code with a synchronous view of handling data; indeed, this is
the way that most protocol code in Python itself is written.  If the
protocol code is executing a read(), however, I want control flow to
return to my main asynchronous control loop while that code is
blocked.  There are, unfortunately, many places executing read() and only
one execution path which knows which one it wishes to yield *to*.

I believe that this is a fairly common application.

The motivation for implementing PEP 255 is exactly this -- a situation
where execution should proceed asynchronously but the application should
be written synchronously for clarity.  The general solution to this
problem is cooperative multithreading.

Benefits to this approach would be: 

 * one-to-many relationships (as described above) are handled as well as
   many-to-one.
 * more pythonic: implement a general approach in the core and build upon
   it rather than a specific approach in the core.

Disadvantages to the PEP 255 approach:

 * we'll probably want microthreads eventually; this feature will bloat
   the language unnecessarily then.
 * less pythonic: once we've implemented generators this way, what about
   simulation loops?  Do we need syntax support for those too?

The one advantage to the PEP 255 way of doing things is that it looks like
it might be slightly easier to implement.  However, C. Tismer has already
shown it's not insurmountable to implement microthreads in Python, and
perhaps some, if not all, of his work could be stolen ;-).

the-view-sure-is-nice-up-here-in-the-peanut-gallery-ly y'rs,

                      ______      __   __  _____  _     _
                     |  ____ |      \_/   |_____] |_____|
                     |_____| |_____  |    |       |     |
                     @ t w i s t e d m a t r i x  . c o m
                     http://twistedmatrix.com/users/glyph