Basic tokenizer

[Andrea Griffini]

On Thu, 2 Sep 2004 08:24:19 +0200, "Miki Tebeka"
<miki.tebeka at zoran.com> wrote:

>There are several parsing/lexing packages for Python. My favorite is PLY
>(http://systems.cs.uchicago.edu/ply/).

I gave it a very quick look and I've to say that I was
impressed... but it wasn't a good impression.

For example the pycalc.py example is just a few lines
shorter than a recursive descent parser that implements
a similar calculator *without importing any module, not
even "re"*. If implementing something from scratch requires
the same amount of code than using a tool I begin wondering
what's the point in using such a tool.

I was told that the good point about parser generators
is that at least you can be sure the code is correct.

Try feeding "10/2" or ")" to pycalc.py ... :-(

Andrea