[Python-Dev] Cython as a Python implementation (was: PEP 3003 - Python Language Moratorium)

[Stefan Behnel]

Michael Foord, 05.11.2009 12:38:
> Stefan Behnel wrote:
>> Michael Foord, 05.11.2009 11:18:
>>> I think the main point is that Cython is not an
>>> implementation of Python. It uses Python syntax, calling into the Python
>>> runtime for many of its features
>>
>> It is a Python implementation in the sense that it implements syntax and
>> semantics of the Python language, be it through generated C code or by
>> calling into the CPython runtime.
> 
> Implementing Python by calling into Python does not seem to really be a
> 'new implementation'... In that sense Cython is a wrapper not an
> implementation.

Would it help anyone if we started forking CPython, just so we can say
"look, we implement Python"? (note that this will never happen, I'm just
trying to find out where we can draw the line here)

To clarify this statement:

>> You could theoretically drop the entire interpreter from CPython, and
>> Cython would still work with the remaining parts

I think the main parts of CPython that we (re-)use are:

- all built-in types
- ref-counting and garbage collection
- frames for traceback reporting
- stdlib

There may be other parts, but that's what comes to my mind right now.

You can additionally use the CPython interpreter through the 'exec'
statement or by calling into uncompiled code.

For a Python implementation, I don't see any use in reimplementing the
built-in types, for example. They are well done and pretty fast, especially
when used from C code, i.e. outside of the interpreter.

The frame/traceback handling code is only for interaction with uncompiled
Python code, so that you get nice looking tracebacks (which I personally
wouldn't consider a required part of the language itself).

Most Python implementations do not reimplement the stdlib, or at most a
minor part of it, so that's right out of the discussion.

I'm not sure how to value the ref-counting/GC reuse, but I doubt that it is
worth being considered an argument for Cython being a CPython wrapper.


>> Programmers can decide themselves if they want to statically type
>> variables, and they can do so without breaking with Python syntax. It is
>> certainly not required to use non-Python syntax to compile Python
>> code, not even if you want to get a fast binary module as a result.
> 
> But you extend the language. Coders can write Python or 'not-Python' (or
> perhaps 'Python plus'). Where you extend the language you are no longer
> implementing Python but a language oriented tool with Python inspired
> syntax intended for use with Python.

But that's just a detail of the parser. It wouldn't be hard at all to
switch off the support for extended features when compiling .py files (as
opposed to .pyx files). It's just not currently done.

Would you consider Cython a Python implementation if we implemented this?
(which I guess we will do anyway at some point, simply because the
extensions are actually invalid code in the given context)

Stefan