[issue15955] gzip, bz2, lzma: add option to limit output size

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Actually it should be:

    # Using zlib's interface
    while not d.eof:
        output = d.decompress(d.unconsumed_tail, 8192)
        while not output and not d.eof:
            compressed = f.read(8192)
            if not compressed:
                raise ValueError('End-of-stream marker not found')
            output = d.decompress(d.unconsumed_tail + compressed, 8192)
        # <process output>

Note that you should use d.unconsumed_tail + compressed as input, and therefore do an unnecessary copy of the data.

Without explicit unconsumed_tail you can write input data in the internal mutable buffer, it will be more effective for large buffer (handreds of KB) and small input chunks (several KB).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue15955>
_______________________________________