[Python-Dev] Using Cython for standard library?

[Stefan Behnel]

Gerhard Häring wrote:
> Cython is still being worked on (intensively, it seems). Bundling it
> with Python means deciding on a particular version probably for an
> entire major release lifecycle (use Cython x.y.{newest} for Python 2.7,
> for example).

Yes, that is definitely an issue. Cython is heavily being worked on, a lot of
features were implemented this summer. This means that even slightly different
Cython versions can generate heavily different code.

The project has made inclusion into Python's stdlib a goal right from the
beginning. However, it is not currently clear when we will reach a suitable
level of Python language compliance to make a language/feature freeze. Note
that this only means a state where we decide to keep new features out. What's
in there now will not go away, so existing Cython code is pretty much future
proof (it even runs on Py3 already :).


>> I would love to see the option to write the lower levels in something
>> other than C, 

Well, Cython is definitely an option for that.


> Absolutely. That's why I tried to reimplement pysqlite in something
> easier to maintain than handwritten Python C API. There's a ctypes-based
> version in its Mercurial repository that's good enough to be used from
> PyPy now. And a started Cython-based one.

ctypes makes sense for projects that do not require a high-speed interface,
i.e. if you do major things behind the interface and only call into it from
time to time, choosing ctypes will keep your code more portable without
requiring a C compiler. However, if speed matters then it's hard to beat
Cython even with hand-written C code.


>> but obviously any choice would have to be a good one.
>> [...]  So, the question I see isn't if Cython should be
>> allowed for standard library modules, but if the landscape of such
>> solutions is at a point that any of them is ready to be committed to.

Well, we had reports that existing, non-trivial C modules could be rewritten
in Cython within a few days. But even if you are only talking about new
modules that get added to the stdlib, all you really need to become
independent of a specific Cython version is a good test suite for your code to
make sure it keeps working as expected when you update your compiler.

Stefan