Real-world Python code 700 times slower than C

[Aahz Maruch]

In article <e2942cb3.0201041647.20271a23 at posting.google.com>,
Brent Burley <brent.burley at disney.com> wrote:
>
>I often use a "10x" rule of thumb for comparing Python to C, but I
>recently hit one real-world case where Python is almost 700 times
>slower than C!  We just rewrote the routine in C and moved on, but
>this has interesting implications for Python optimization efforts.

Yes, tight loops in Python have extremely high overhead relative to C.

>As an aside, there's another interesting bottleneck we hit in our
>production code.  We're reading a lookup table from a text file (for
>doing image display color correction) that consists of 64K lines with
>3 integers on each line.  The python code looks something like:
>
>rArray = []
>gArray = []
>bArray = []
>for line in open(lutPath).xreadlines():
>    entry = split(line)
>    rArray.append(int(entry[0]))
>    gArray.append(int(entry[1]))
>    bArray.append(int(entry[2]))

Out of curiosity, given that this is apparently something you need to do
frequently, why not optimize by doing a "cache" pass to another file
that's optimized for reading in Python (perhaps as a pickle)?  That way
you only pay a huge penalty once per file update.  In addition, make
sure you're using Python >= 2.1.1; there were some file reading
enhancements that made I/O several times faster.

>From my perspective, the joy of Python is the freedom it gives you for
looking at brand-new ways of solving problems.
-- 
                      --- Aahz  <*>  (Copyright 2002 by aahz at pobox.com)

Hugs and backrubs -- I break Rule 6                 http://www.rahul.net/aahz/
Androgynous poly kinky vanilla queer het Pythonista   

Opinions are free.  Go ahead, take one.