How slow? (was Re: Python IS slow !)

[Robb Shecter]

Gordon McMillan wrote:

> Most of the specific complaints about Python's slowness come from
> people doing regular expression searches of large text files and
> comparing the speed with Perl. No question, Perl wins.
>

...Although, not by enough to it worth programming in perl:

I have a program that pre-processes web server log files for me before
I run them through Analog.  I wrote it three times - in perl, python and
java - as a speed and style comparison.

The program basically gets rid of junk in the file that would through off
statistics - it checks for likely web spiders, and filters out those
entries.  It filters out any entries I cause while checking the site.
So, by nature it does lots of regex work on large (multi-megabyte) files.

At first my perl program was 3x faster than the other two, but it was a
typical quick perl hack:

while(<>) {
  next if /----/;
  ...
  print;
}

...as my requirements got more complex I rewrote it to be more modular,
using data structures and function calls.  That really bumped up the perl
execution time.  Now the speed of the three programs is very close -
close enough that the coding style of python or java outweighs any
benefit of perl's speed.  I haven't done strict tests - just observed the
programs in real-world use.

- Robb