Stackless & String-processing

[Neel Krishnaswami]

In article <378A40AD.B4FDB79F at appliedbiometrics.com>,
Christian Tismer  <tismer at appliedbiometrics.com> wrote:
>
>               Stackless Python 0.3
>
>The possibilities are for instance:
>
>Continuations, with which we will build Coroutines and Generators
>
>Coroutines are able to run at the speed of a single C function call,
>which makes them a considerable alternative in certain algorithms.
>This is no longer a speculation since I have working coroutine
>prototypes, written with continuations.

I've been looking at Icon, and it occurs to me that if coroutines and
generators were available at the Python level, it might yield a way of
doing string processing that is more "Pythonic" than regexps.

Regexps are nice because when you have a pattern that they can
directly represent, then you can simply specify a pattern and then you
don't have to worry about the tedious bookkeeping of looping over the
string and keeping track of the state variable.

However, when you need to match a pattern that a regexp can't match,
then suddenly you need to break the string into tokens with regexps
and then loop over the pieces and keep track of a bunch of state
variables that don't necessarily correspond to the pieces you are
actually interested in.

This is unpleasant, because a) clumsy bookkeeping is bad, and b)
there's two different sorts of syntax needed to do basically 
similar tasks. If we could compose generators just like functions,
then the bookkeeping can be abstracted away and the same approach
will work for arbitrarily complicated parsing tasks. 

So I think it would be nice if these lovely toys were available at
the Python level. Is this a possibility?

(I'd love to be able to define coroutines in Python like so:

def evens(z):
    for elt in z:
        if z % 2 == 0:
            suspend(z)

It would probably require some rethinking of Python's iteration
protocol though.)


Neel