[Python-Dev] PEP 574 -- Pickle protocol 5 with out-of-band data
Pierre Glaser
pierre.glaser at inria.fr
Mon Nov 5 06:36:50 EST 2018
Hi All,
As part of our scikit-learn development and our effort to provide better
parallelism for python, we rely heavily on dynamic classes and functions
pickling. For this
usage we use cloudpickle, but it suffers from performance issues due to
its pure python implementation. After long discussions with Olivier
Grisel and Thomas
Moreau, we ended up agreeing on the fact that the best solution for this
problem would be to add those functionalities to the _pickle.c module.
I am already quite familiar with the C/Python API, and can dedicate a
lot of my time in the next couple months to make this happen.
Serhiy, from this thread
(https://mail.python.org/pipermail/python-dev/2018-March/152509.html),
it seems that you already started implementing local classes pickling.
I would be happy to use this work as a starting point and build from it.
What do you think?
Regards,
Pierre
More information about the Python-Dev
mailing list