[Python-Dev] Cython for cPickle?

[Antoine Pitrou]

On Thu, 19 Apr 2012 10:55:24 +0200
Stefan Behnel <stefan_ml at behnel.de> wrote:
> 
> I noticed that there is a PEP (3154) and a GSoC proposal about improving
> Pickle. Given the recent discussion on this list about using Cython for the
> import module, I wonder if it wouldn't make even more sense to switch from
> a C (accelerator) implementation to Cython for _pickle.

I think that's quite orthogonal to PEP 3154 (which shouldn't add a lot
of new code IMHO).

> Note that the approach won't be as simple as compiling pickle.py. _pickle
> uses a lot of optimisations that only work at the C level, at least
> efficiently. So the idea would be to rewrite _pickle in Cython instead.
> It's currently about 6500 lines of C. Even if we divide that only by a
> rather conservative factor of 3, we'd end up with some 2000 lines of Cython
> code, all extracted straight from the existing C code. That sounds like
> less than two weeks of work, maybe even if we add the marshal module to it.

I think this all needs someone to demonstrate the benefits, in
terms of both readability/maintainability, and performance.

Also, while C is a low-level language, Cython is a different language
than Python when you start using its optimization features. This means
core developers have to learn that language.

Regards

Antoine.