[issue16350] zlib.Decompress.decompress() after EOF discards existing value of unused_data

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

>From Nadeem Vawda's mail:

> I wasn't suggesting that you try to resize the existing unused_data
> object; I agree that this would be a bad idea. What I was thinking of
> was something like this:
> 
>     size_t old_unused_size = PyBytes_GET_SIZE(self->unused_data);
>     size_t new_unused_size = old_unused_size + self->zst.avail_in;
>     PyObject *new_unused_data = PyBytes_FromStringAndSize(NULL, 0);
>     if (_PyBytes_Resize(&new_unused_data, new_unused_size) < 0) {
>         Py_DECREF(RetVal);
>         goto error;
>     }
>     memcpy(PyBytes_AS_STRING(new_unused_data),
>            PyBytes_AS_STRING(self->unused_data),
>            old_unused_size);
>     memcpy(PyBytes_AS_STRING(new_unused_data) + old_unused_size,
>            self->zst.next_in, self->zst.avail_in);
>     Py_DECREF(self->unused_data);
>     self->unused_data = new_unused_data;
> 
> Basically, hacking around the fact that (AFAICT) there's no direct way to
> allocate an uninitialized bytes object. That way we only do one allocation,
> and only copy the new data once.

This hacking is not needed, if first argument of PyBytes_FromStringAndSize() is NULL, the contents of the bytes object are uninitialized. And more, this hacking is invalid, because empty bytes object shared and can't be resized.

Patch updated. The concatenation optimized as you suggested, one bug fixed.

----------
Added file: http://bugs.python.org/file27775/zlib_accum_unused_data_2.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16350>
_______________________________________