[Tutor] Reading large bz2 Files

[Norman Rieß]

Am 19.02.2010 22:03, schrieb Kent Johnson:
> On Fri, Feb 19, 2010 at 7:42 AM, Norman Rieß <norman at smash-net.org> wrote:
>   
>> Hello,
>>
>> i am trying to read a large bz2 file with this code:
>>
>> source_file = bz2.BZ2File(file, "r")
>> for line in source_file:
>>    print line.strip()
>>
>> But after 4311 lines, it stoppes without a errormessage. The bz2 file is
>> much bigger though.
>> How can i read the whole file line by line?
>>     
> I wonder if it is dying after reading 2^31 or 2^32 bytes? It sounds a
> bit like this (fixed) bug:
> http://bugs.python.org/issue1215928
>
> Kent
>
>   
./osmcut.py ../planet-100210.osm.bz2 > test.txt
smash at loki ~/osm/osmcut $ ls -lh test.txt
-rw-r--r-- 1 871K 19. Feb 22:41 test.txt

Seems like far from it.

Norman