mxTools (was Re: why no "do : until"?)

[Paul Prescod]

Mike Fletcher wrote:
> 
> .. mxTextTools can do
> that scripting using its state-transition table definitions, and as a
> result, can run the entire parsing process in C .

Cna mxTextTools work with streams without reading them all into memory?
I find it quite a pain to use hand-crafted RE-based lexers on streaming
information. 

I haven't tried this recently but if I remember correctly, the problem
is: How would I do a regexp match on "Py..n" on a 1-gigabyte file with a
fixed amount of "lookahead"? Do I have to load in chunks at a time and
then do the pattern matching across "chunk boundaries" myself? If
mxTextTools handles this, I would see that as a big advantage.

 Paul Prescod