Perl speed vs. Python speed

[Tom Culliton]

In article <85hjfa$mlp$1 at nnrp1.deja.com>,  <rwgk at my-deja.com> wrote:
>I just wrote a tiny Python program to do some text processing.
>When I ran it I was very surprised how long it took to finish.
>My first guess is that Perl would be a lot faster, maybe even a
>factor 10, but I did not do a direct comparison.
>
>Could someone with more experience comment on Perl speed vs.
>Python speed?

Naively written Python is often not as fast as Perl for processing
large quantities of text.  On the other hand you'll still be be able
to read and understand the Python 6 months from now, and by using some
common optimizaions it you can be speed up considerably while still
remaining readable.

Things to look out for:

Reading a single line at a time in Python is not as fast as it could
be.  Using readlines with or without a buffer size argument to limit
memory usage can make a huge difference.

Function calls and name look up outside the local scope are relatively
expensive.  Using the localization trick can make an improvement of
several percent per localized variable, as can moving function code
in line.

The "new" Python re module needs to be optimized and possibly pushed
into C code, the older regex module is about twice as fast.

Precomputing values like bound regex search or match functions can
also help quite a bit.

Python can spend a lot of time allocating and freeing memory if you
get sloppy with your string manipulations.

There are other tricks but this is just supposed to be a quick answer.
Using the profiler is often very instructive.  Posting some code might
also get you more examples.

Bottom line, Perl does nasty incestuous things with i/o and regular
expressions to make them very fast.  Python does less magic, which
means that it leaves more room for you to do things like provide
pseudo file objects with special behaviour (like the StringIO module),
or replace the regular expression module (regex vs. re).