Python example: possible speedup?

[Skip Montanaro]

    Hrvoje> The program was quite easy to write, and easy to read
    Hrvoje> afterwards.  The problem is that it is also quite slow.  On my
    Hrvoje> system, it takes about 27 CPU seconds (as reported by `time'
    Hrvoje> shell builtin) to do the work, which can extend to more than a
    Hrvoje> minute of real time, depending on the system load.

        ...

    Hrvoje> I would really appreciate some suggestions.  The code is not
    Hrvoje> large, and is (I hope) rather elegant.  I am a Python beginner,
    Hrvoje> so I'd also appreciate tips on Python style and OO technique.
    Hrvoje> I'll post/mail the Perl equivalent on demand.

Assuming the package files aren't humongous, I'd suggest your __init__
method just slurp up the entire file instead of reading it line-by-line in
the next_header method.  As has been recounted here before, Python does not
go to great lengths to optimize I/O the way Perl has, instead just relying
on a fairly straightforward interface to stdio.

Also, instead of calling next_header on a line-by-line basis, how about
calling it once per package and returning a dict containing the header names
as keys?  Again, function calling in Python, while flexible, isn't terribly
efficient.  A common optimization people use is to operate on aggregate
quantities in functions instead of their individual elements.

Skip Montanaro | http://www.mojam.com/
skip at mojam.com | http://www.musi-cal.com/~skip/
847-971-7098   | Python: Programming the way Guido indented...