Regex Speed

[Pop User]

garrickp at gmail.com wrote:
> While creating a log parser for fairly large logs, we have run into an
> issue where the time to process was relatively unacceptable (upwards
> of 5 minutes for 1-2 million lines of logs). In contrast, using the
> Linux tool grep would complete the same search in a matter of seconds.
>   
Its very hard to beat grep depending on the nature of the regex you are 
searching using. The regex engines in python/perl/php/ruby have traded 
the speed of grep/awk for the ability to do more complex searches.

http://swtch.com/~rsc/regexp/regexp1.html

This might not be your problem but if it is you can always popen grep.

It would be nice if there were a Thompson NFA re module.