Comparison of parsers in python?

[Peng Yu]

On Sun, Sep 20, 2009 at 8:19 AM, andrew cooke <andrew at acooke.org> wrote:
>
> also, parsing large files may be slow.  in which case you may be
> better with a non-python solution (even if you call it from python).
>
> your file format is so simple that you may find a lexer is enough for
> what you want, and they should be stream oriented.  have a look at the
> "shlex" package that is already in python.  will that help?
>
> alternatively, perhaps plex - http://www.cosc.canterbury.ac.nz/greg.ewing/python/Plex/
> - that is pure python, but greg ewing is a good programmer and he says
> on that page it is as fast as possible for python, so it is probably
> going to be quite fast.
>
> andrew
>
> ps maybe you already know, but a lexer is simpler than a parser in
> that it doesn't use the context to decide how to treat things.  so it
> can recognise something is a number, or a word, or a quoted string,
> but not whether it is part of a track definition line or a data value,
> for example.  but in this case the format is so simple that a lexer
> might do quite a ot of what you want, and would make the remaining
> plain python program very simple.

I don't quite understand this point.  If I don't use a parser, since
python can read numbers line by line, why I need a lexer package?

Regards,
Peng