PyLR -- Fast LR parsing in python

What is PyLR?
What is the current state of PyLR?
Where do I get PyLR?
What will be added to PyLR?
Where do I find out about parsing theory?
How can I contribute to PyLR?

What is PyLR?

PyLR is a package of tools for creating efficient parsers in python, commonly known as a compiler compiler. PyLR is currently under development. Many of the tools programmers will need are already there. However, there is a lot more that can be added.

PyLR (pronounced 'Pillar') was motivated by the frequencly with which parsers are hand coded in python, the performance demands that these parsers are subject to (you just can't beat native machine code for speed...), and academic curiosity (I wanted to really know how LR parsing works).

What is the current state of PyLR?

PyLR currently has class interfaces to a Grammar, a Lexer, and an extension module defining a parsing engine builtin type. All of these components are based on sound parsing theory, but nevertheless haven't been tested by anyone but it's author. The code as is stands can definitely be of use to anyone hand writing a parser, but some of the nicer things in the complete package just haven't been done yet .

PyLR is therefore under development, as it will always be. PyLR will be given a release number once it supplies the following tools:

write an 'engine' module that implements the LR parsing algorythm in C with callbacks to python functions. (done)
write a Lexer class using re (done)
write a Grammar class that will take as input a context free grammar and produce the parsing tables necessary to complement the engine. This is to be done with LR(1) grammars (done -- extremely inefficient) and LALR(1) Grammars(done, except with epsilon productions, much more efficient).
add a user interface -- manually write a lexer and Grammar using the exisiting classes to parse lexer and grammar specifications modelled after lex/flex and yacc/bison. (done for Grammars)
write documentation. (usable, but not done)
(post release) add grammars to various languages to the distribution.

These things will probably be done over the next month or two (as I only have free time to give to this project...Ahemmm...).

Where do I get PyLR?

You can get PyLR in one of two places, here or here. Both versions will be in sync with each other.

What will be added to PyLR?

In addition to the list of things to finish before a full release, is published, PyLR could be used as the basis for an efficient datapath analyzer (optimizer), for a front end to translation from one language to another, for type checking code, etc.

As soon as the first release is completed, Tools to aid in all these things could well be added to the package. Also, anyone wanting to contribute parser specifications for languages of general use is most welcome.

Where do I find out more about parsing?

Parsing was for a long time a big challenge for computer scientists. The need for computer parsing originally came about with the first writing of compilers. Since then, the theory behind parsing has been studied in depth and has pretty much stabilized as it no longer really presents a big problem in terms of speed or size in terms of parsing todays computer languages. One standard means of parsing that has been used for years because of its efficiency is LR parsing (more particularly, LALR parsing). A lot of good information is in Lex and Yacc , The Dragon Book , and it seems like the only place to find good info on LALR parsing is in

"Efficient Computation of LALR(1) Look-Ahead sets",
ACM "Transactions on Programming Languages and Syntax", Vol 4 No 4, 1982

Finally, to find out how to use PyLR, see thePyLR manual

How do I contribute to PyLR?

mail me.