speed problems

[Antonio Cavallo]

> I've become interested in Python a while ago and just converted a simple
> perl script to python.
> I've found that there's a huge difference in execution time for the scripts,
> in favor of perl and I can't pinpoint what's going wrong;

I had the same problem in dealing with a large (compressed) file using
python vs c++ (using the gzip library to open/read a file): the
results were in favour of python against c++ this time;)

I think the problem is in the flow: 

gzip -> file/pipe -> perl
file -> zlib -> python

The decompression through zlib is wfar slower because it is carried
using data chunks: there is no mean to control where a chunk will
terminate (like in correspondence to an EOL).
Try using: zcat <myfile> | myprogram.py
In my case it solved all the puzzling speed problems: zcat
uncoditionally dumps the data irregarding where the EOL happens (so
the upper layer should not wait for the next chunk to be
decompressed).
regards,
antonio cavallo