Basic tokenizer
Dale Strickland-Clak
dale at riverhall.NOSPAM.co.uk
Wed Sep 1 09:05:49 EDT 2004
I'm looking for a simple string tokenizer. The Python tokenize module is
too specific and is geared to Python syntax.
I just want something that can be setup to read the basic stuff like
. identifiers
. integers (and/or reals)
. strings
. special characters such as operators and brackets.
Is anyone aware of such a module?
Thanks.
--
Dale Strickland-Clark
Riverhall Systems Ltd, www.riverhall.co.uk
More information about the Python-list
mailing list