[issue27130] zlib: OverflowError while trying to compress 2^32 bytes or more

[Martin Panter]

Martin Panter added the comment:

Here is a possible patch for 2.7. To fix everything on 2.7 I changed the module to accept input buffers over 2 GiB by enabling PY_SSIZE_T_CLEAN. As a consequence, the patch also includes ports of Nadeem Vawda’s adler32(), crc32() changes from Issue 10276, and my tests from Issue 25626.

I have only tested this on computers with less than 4 GiB of memory. I can test compression and checksums with more input by using a sparse memory map, but not decompression.

fm = open("5GiB.sparse", "w+b")
fm.truncate(5 * 2**30)
m = mmap(fm.fileno(), 0)
z = compress(m)

----------
type:  -> crash
Added file: http://bugs.python.org/file43805/64bit_support_for_zlib_v10-2.7.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue27130>
_______________________________________