Basic tokenizer

[Miki Tebeka]

Hello Dale,

> I'm looking for a simple string tokenizer. The Python tokenize module is 
> too specific and is geared to Python syntax.
> 
> I just want something that can be setup to read the basic stuff like 
> . identifiers 
> . integers (and/or reals)
> . strings
> . special characters such as operators and brackets.
> 
> Is anyone aware of such a module?
> 
> Thanks.
There are several parsing/lexing packages for Python. My favorite is PLY
(http://systems.cs.uchicago.edu/ply/).

Bye.
--
------------------------------------------------------------------------
Miki Tebeka <miki.tebeka at zoran.com>
http://tebeka.spymac.net
The only difference between children and adults is the price of the toys