Ann: Plex 0.1 - A Lexical Analysis Module

[Neel Krishnaswami]

Greg Ewing <greg at cosc.canterbury.ac.nz> wrote:
> Having spent a couple of years wondering "when is someone
> going to write a decent lexing module for Python", I finally
> decided to do it myself.

Very slick, especially the the constructor-style you use with the
FSMs. A first look at the code suggests that unlike with regexps, you
could build symbolic lexers for /arbitrary sequences of Python
objects/.[*]

This is cool all by itself -- think of writing a readable, pure-Python
peephole optimizer with bytecodehacks -- but if you could combine this
with, say, list comprehensions, I think that whole new vistas may open
up....

[*] I think it would be necessary to rewrite TokenStream, but that's
not too bad.


Neel