read large zip file
John Nagle
nagle at animats.com
Mon Apr 7 23:10:01 EDT 2008
Gabriel Genellina wrote:
> En Sun, 06 Apr 2008 19:20:31 -0300, Brian Blais <bblais at bryant.edu>
> escribió:
>
>> I need to read a series of large zipfiles (which only contain one
>> large text file), and I noticed that the zipfile module:
>>
>> 1) has a read method which isn't an iterator, and returns the entire
>> file selected all at once
>> 2) has no readlines method, and no obvious way to implement one
>>
>> Is there a way to stream an unzip, so it behaves more like a file?
>
> Use the module from the 2.6 version; it appears to work fine even on
> Python 2.4 (see this thread
> http://groups.google.com/group/comp.lang.python/browse_thread/thread/71c4890cefac82aa/
> )
It's easier than that:
fd = gzip.open(filename, 'rb')
for line in fd :
processline(line)
This works even in Python 2.4. I use this routinely for processing big
log files.
John Nagle
More information about the Python-list
mailing list