Speed problems with Python vs. Perl

[Tim Hochberg]

> Last week I wrote a simple Python program and found out that it was
> terribly slow. Therefore I retried with Perl and found a much better
> performance. The programs simple read a file and split the lines on white
> space, something I have to do very often for data elaboration

[SNIP]

> As you can see, without the splitting (only reading and counting the
lines)
> Perl is twice as fast, but once I split on whitespace, Python gets more
> than 5 times slower than Perl.
>
> These results have been achieved on a AMD with 650 MHerz. On my home
> machine, a 266 Celeron, the performance gap is worse. Without line
> splitting the programs
> are about the same speed, but with splitting enabled, the Python version
> becomes 10 times slower than the Perl version (5 secs against 0.5 secs)

Why the Perl's REs are faster, I'll leave for someone else to explain.
Perhaps they special case common REs more than Python does. However, you are
suffering from Overuse of Regular Expressions (ORE [tm]). Specifically, you
can use string.split or in Python 2.0 you can use the split method on
strings:

def main152(): # Should work on python 1.5.2
    import string
    icount = 0
    for line in sys.stdin.readlines():
        icount = icount + 1
        f = string.split(line)
    print "Total lines read: " + `icount`

or

def main20(): # Should work on python >= 2.0
    icount = 0
    for line in sys.stdin.readlines():
        icount += 1
        f = line.split()
    print "Total lines read: " + `icount`

or

def main21(): # Should? work on Python >= 2.1
    icount = 0
    for line in sys.stdin.xreadlines(): # Might speed things up a bit....
        icount += 1
        f = line.split()
    print "Total lines read: " + `icount`

I haven't actually benchmarked (or even tried) any of these, but I'd be
curious to see how the last one in particular stacks up against it's Perl
equivalent. I don't have Perl installed at the moment, though so I leave
that as an excercise for the reader....

-lazy tim