[Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Nathaniel Smith]

On Fri, Nov 9, 2018 at 6:03 PM, Victor Stinner <vstinner at redhat.com> wrote:
> Le sam. 10 nov. 2018 à 02:50, Nathaniel Smith <njs at pobox.com> a écrit :
>> Doesn't this mean that you're just making the C API larger and more
>> complicated, rather than simplifying it? You cite some benefits
>> (tagged pointers, changing the layout of PyObject, making PyPy's life
>> easier), but I don't see how you can do any of those things so long as
>> the current C API remains supported.
>
> Tagged pointers and changing the layout of PyObject can only be
> experimented in a new different Python runtime which only supports C
> extensions compiled with the new C API. Technically, it can be CPython
> compiled with a different flag, as there is already python3-dbg (debug
> mode, ./configure --with-pydebug) and python3 (release mode). Or it
> can be CPython fork.
>
> I don't propose to experiment tagged pointer or changing the layout of
> PyObject in CPython. It may require too many changes and it's unclear
> if it's worth it or not. I only propose to implement the least
> controversial part of the new C API in the master branch, since
> maintaining this new C API in a fork is painful.
>
> I cannot promise that it will make PyPy's life easier. PyPy developers
> already told me that they already implemented the support of the
> current C API. The promise is that if you use the new C API, PyPy
> should be more efficient, because it would have less things to
> emulate. To be honest, I'm not sure at this point, I don't know PyPy
> internals. I also know that PyPy developers always complain when we
> *add new functions* to the C API, and there is a non-zero risk that I
> would like to add new functions, since current ones have issues :-) I
> am working with PyPy to involve them in the new C API.

So is it fair to say that your plan is that CPython will always use
the current ("old") API internally, and the "new" API will be
essentially an abstraction layer, that's designed to let people write
C extensions that target the old API, while also being flexible enough
to target PyPy and other "new different Python runtimes"?

If so, then would it make more sense to develop this as an actual
separate abstraction layer? That would have the huge advantage that it
could be distributed and versioned separately from CPython, different
packages could use different versions of the abstraction layer, PyPy
isn't forced to immediately add a bunch of new APIs...

-n

-- 
Nathaniel J. Smith -- https://vorpus.org