really slow gzip decompress, why?

redbaron ivanov.maxim at gmail.com
Mon Jan 26 10:22:25 EST 2009


I've one big (6.9 Gb) .gz file with text inside it.
zcat bigfile.gz > /dev/null does the job in 4 minutes 50 seconds

python code have been doing the same job for 25 minutes and still
doesn't finish =( the code is simpliest I could ever imagine:

def main():
  fh = gzip.open(sys.argv[1])
  all(fh)

As far as I understand most of the time it executes C code, so pythons
no overhead should be noticible. Why is it so slow?



More information about the Python-list mailing list