Why is text file processing SO slow?

[David Bolen]

jaimewyant at my-deja.com writes:

> The above algorithm is only slightly stripped down.  The above program
> written in Python takes 55 seconds to execute.  The same program
> written in perl takes 15 seconds.  The C version takes 10 seconds.
> 
> Is it typically a BAD idea to use Python (1.5.2) to process text files?

I certainly wouldn't say it's a bad idea, but it is true that raw text
processing is an area where Perl can pretty easily exceed Python in
performance.  However, in the cases I've worked with it hasn't been as
extreme as what you are showing.  While others have pointed out other
items, I'm wondering if you might include the actual code (not
pseudo-code) that you are using for Perl and Python - there might be
some optimizations possible.  (For example, are you pre-compiling the
regex in Python?)

--
-- David
-- 
/-----------------------------------------------------------------------\
 \               David Bolen            \   E-mail: db3l at fitlinxx.com  /
  |             FitLinxx, Inc.            \  Phone: (203) 708-5192    |
 /  860 Canal Street, Stamford, CT  06902   \  Fax: (203) 316-5150     \
\-----------------------------------------------------------------------/