CRC-checksum failed in gzip

Laszlo Nagy gandalf at shopzeus.com
Wed Aug 1 13:57:19 EDT 2012


>> Make sure that file objects are not shared between threads. If that is
>> possible. It will probably solve the problem (if that is related to
>> threads).
>
> Well I just have to create a lock I guess right?
That is also a solution. You need to call file.read() inside an acquired 
lock.
> with lock:
>      # open file
>      # read content
>
But not that way! Your example will keep the lock acquired for the 
lifetime of the file, so it cannot be shared between threads.

More likely:

## Open file
lock = threading.Lock()
fin = gzip.open(file_path...)
# Now you can share the file object between threads.

# and do this inside any thread:
## data needed. block until the file object becomes usable.
with lock:
     data = fin.read(....) # other threads are blocked while I'm reading
## use your data here, meanwhile other threads can read





More information about the Python-list mailing list