[Python-Dev] PEP 574 (pickle 5) implementation and backport available

[Raymond Hettinger]

> On May 24, 2018, at 10:57 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> While PEP 574 (pickle protocol 5 with out-of-band data) is still in
> draft status, I've made available an implementation in branch "pickle5"
> in my GitHub fork of CPython:
> https://github.com/pitrou/cpython/tree/pickle5
> 
> Also I've published an experimental backport on PyPI, for Python 3.6
> and 3.7.  This should help people play with the new API and features
> without having to compile Python:
> https://pypi.org/project/pickle5/
> 
> Any feedback is welcome.

Thanks for doing this.

Hope it isn't too late, but I would like to suggest that protocol 5 support fast compression by default.  We normally pickle objects so that they can be transported (saved to a file or sent over a socket). Transport costs (reading and writing a file or socket) are generally proportional to size, so compression is likely to be a net win (much as it was for header compression in HTTP/2).

The PEP lists compression as a possible a refinement only for large objects, but I expect is will be a win for most pickles to compress them in their entirety.


Raymond