[Python-Dev] PEP 399: Pure Python/C Accelerator Module Compatibiilty Requirements

[Stefan Behnel]

Brett Cannon, 05.04.2011 01:46:
> At both the VM and language summits at PyCon this year, the issue of
> compatibility of the stdlib amongst the various VMs came up. Two issues came
> about in regards to modules that use C code. One is that code that comes in
> only as C code sucks for all other VMs that are not CPython since they all
> end up having to re-implement that module themselves. Two is that modules
> that have an accelerator module (e.g., heapq, warnings, etc.) can end up
> with compatibility options (sorry, Raymond, for picking on heapq, but is was
> what bit the PyPy people most recently =).
>
> In lieu of all of this, here is a draft PEP to more clearly state the policy
> for the stdlib when it comes to C code. Since this has come up before and
> this was discussed so much at the summits I have gone ahead and checked this
> in so that even if this PEP gets rejected there can be a written record as
> to why.
>
> And before anyone asks, I have already run this past the lead devs of PyPy,
> Jython, and IronPython and they all support what this PEP proposes. And with
> the devs of the other VMs gaining push privileges there shouldn't be an
> added developer burden on everyone to make this PEP happen.

This PEP has received a lengthy discussion by now, so here's why I think 
it's being fought so heavily by several CPython core developers, 
specifically those who have traditionally carried a large part of the 
optimisation load in the project.

I think the whole point of this PEP is that, having agreed that a shared 
standard library for all Python implementations is a good thing, the amount 
of shareable code should be maximised. I doubt that anyone will argue 
against this goal.

But that obviously includes all sides. If other implementations are free to 
cherry pick the targets of their own effort geared by the optimisation of 
their own implementation, and leave the whole burden of compatibility and 
code reusability on CPython, in addition to the CPython efforts of 
improving and optimising its own core code base and its own stdlib version, 
it's not an equal matter.

That's what makes the PEP feel so unfair to CPython developers, because 
they are the ones who carry most of the burden of maintaining the stdlib in 
the first place, and who will most likely continue to carry it, because 
other implementations will continue to be occupied with their own core 
development for another while or two. It is nice to read that other 
implementations are contributing back patches that simplify their own reuse 
of the stdlib code. However, that does not yet make them equal contributors 
to the development and the maintenance of the stdlib, and is of very little 
worth to the CPython project. It often even runs counter to the interest of 
CPython itself.

I think this social problem of the PEP can only be solved if the CPython 
project stops doing the major share of the stdlib maintenance, thus freeing 
its own developer capacities to focus on CPython related improvements and 
optimisations, just like the other implementations currently do. I'm not 
sure we want that at this point.

Stefan