[Python-Dev] PEP 574 (pickle 5) implementation and backport available

[Neil Schemenauer]

On 2018-05-25, Antoine Pitrou wrote:
> Do you have something specific in mind?

I think compressed by default is a good idea.  My quick proposal:

- Use fast compression like lz4 or zlib with Z_BEST_SPEED

- Add a 'compress' keyword argument with a default of None.  For
  protocol 5, None means to compress.  Providing 'compress' != None
  for older protocols will raise an error.

The compression overhead will be small compared to the
pickle/unpickle costs.  If someone wants to apply their own (e.g.
better) compression, they can set compress=False.

An alternative idea is to have two different protocol formats.  E.g.
5 and 6.  One is "pickle 5" with compression, one without
compression.  I don't like that as much since it breaks the idea
that higher protocol numbers are "better".

Regards,

  Neil