[Python-Dev] Using Cython for standard library?

[Gerhard Häring]

Calvin Spealman wrote:
> On Mon, Nov 3, 2008 at 6:29 AM, Gerhard Häring <gh at ghaering.de> wrote:
>> Stefan Behnel wrote:
>>> Michael Foord wrote:
>>>> Moving more C extensions to an implementation based on ctypes would be
>>>> enormously useful for PyPy, IronPython and Jython, but ctypes is not yet
>>>> as portable as Python itself which could be an issue (although one worth
>>>> resolving).
>>> In the same line, moving more extensions to a high-level language like
>>> Cython,
>>> instead of writing them in straight C, would make a later switch to a
>>> different environment a lot easier, as the extensions could be regenerated
>>> with a modified Cython compiler (obviously minus some fixing of premature
>>> optimisations and the like).
>> Is using Cython for anything in Modules/ really an option? In my limited
>> experiments with it, I did like it.
>>
>> But using it for Python standard library stuff doesn't look quite right to
>> me:
>>
>> - Option 1: distribute Cython with Python and integrate into build process
>> -- Ouch!
> 
> Can you be a bit more descriptive?

Cython is still being worked on (intensively, it seems). Bundling it 
with Python means deciding on a particular version probably for an 
entire major release lifecycle (use Cython x.y.{newest} for Python 2.7, 
for example).

>> - Option 2: only distribute generated source files
>> -- developers still need to have Cython installed
>> -- you have to trust Cython; who will really review the generated code?
> 
> Who reviews the machine code from gcc?

That's comparing apples and eggs :-P But it may be that I'm a little 
paranoid here.

> I would love to see the option to write the lower levels in something
> other than C, 

Absolutely. That's why I tried to reimplement pysqlite in something 
easier to maintain than handwritten Python C API. There's a ctypes-based 
version in its Mercurial repository that's good enough to be used from 
PyPy now. And a started Cython-based one.

> but obviously any choice would have to be a good one.
> [...]  So, the question I see isn't if Cython should be
> allowed for standard library modules, but if the landscape of such
> solutions is at a point that any of them is ready to be committed to.

ACK.

-- Gerhard