[Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data

Pierre Glaser pierre.glaser at inria.fr
Mon Nov 5 06:36:50 EST 2018


Hi All,
As part of our scikit-learn development and our effort to provide better 
parallelism for python, we rely heavily on dynamic classes and functions 
pickling. For this
usage we use cloudpickle, but it suffers from performance issues due to 
its pure python implementation. After long discussions with Olivier 
Grisel and Thomas
Moreau, we ended up agreeing on the fact that the best solution for this 
problem would be to add those functionalities to the _pickle.c module.
I am already quite familiar with the C/Python API, and can dedicate a 
lot of my time in the next couple months to make this happen.

Serhiy, from this thread 
(https://mail.python.org/pipermail/python-dev/2018-March/152509.html), 
it seems that you already started implementing local classes pickling.
I would be happy to use this work as a starting point and build from it. 
What do you think?

Regards,

Pierre



More information about the Python-Dev mailing list