[Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data

[Serhiy Storchaka]

28.03.18 21:39, Antoine Pitrou пише:
 > I'd like to submit this PEP for discussion.  It is quite specialized
 > and the main target audience of the proposed changes is
 > users and authors of applications/libraries transferring large amounts
 > of data (read: the scientific computing & data science ecosystems).

Currently I'm working on porting some features from cloudpickle to the 
stdlib. For these of them which can't or shouldn't be implemented in the 
general purpose library (like serializing local functions by serializing 
their code objects, because it is not portable) I want to add hooks that 
would allow to implement them in cloudpickle using official API. This 
would allow cloudpickle to utilize C implementation of the pickler and 
unpickler.

There is a private module _compat_pickle for supporting compatibility of 
moved stdlib classes with Python 2. I'm going to provide public API that 
would allow third-party libraries to support compatibility for moved 
classes and functions. This could also help to support classes and 
function moved in the stdlib after 3.0.

It is well known that pickle is unsafe. Unpickling untrusted data can 
cause executing arbitrary code. It is less known that unpickling can be 
made safe by controlling resolution of global names in custom 
Unpickler.find_class(). I want to provide helpers which would help 
implementing safe unpickling by specifying just white lists of globals 
and attributes.

This work still is not finished, but I think it is worth to include it 
in protocol 5 if some features will need bumping protocol version.