[Python-Dev] Re: Automatic flex interface for Python?

[Tim Peters]

[Eric S. Raymond]
> ...
> Lexers are painful in Python.

This is so.

> They hit the language in a weak spot created by the immutability of
> strings.

But you lost me here -- I don't see a connection between immutability and
either ease of writing lexers or speed of lexers.  Indeed, lexers are (IME)
more-or-less exactly as painful and slow written in Perl, where strings are
mutable.

Seems more to me that lexing is convenient and fast only when expressed in a
language specifically designed for writing lexers, and backed by a
specialized execution engine that knows a great deal about fast
state-machine implementation.  Like, say, Flex.  Lexing is also clumsy and
slow in SNOBOL4 and Icon, despite that they excel at higher-level
pattern-matching tasks.  IOW, lexing is in a world by itself, almost nothing
is good at it, and the few things that shine at it don't do anything else.

lexing-is-the-floating-point-execution-unit-of-the-language-world-ly
    y'rs  - tim