[Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

[Victor Stinner]

To hide all implementation details, I propose to stop using macros and
use function calls instead. For example, replace:

#define PyTuple_GET_ITEM(op, i) \
   (((PyTupleObject *)(op))->ob_item[i])

with:

# define PyTuple_GET_ITEM(op, i) PyTuple_GetItem(op, i)

With this change, C extensions using PyTuple_GET_ITEM() does no longer
dereference PyObject* nor access PyTupleObject.ob_item. For example,
PyPy doesn't have to convert all tuple items to PyObject, but only
create one PyObject for the requested item. Another example is that it
becomes possible to use a "CPython debug runtime" which checks at
runtime that the first argument is a tuple and that the index is
valid. For a longer explanation, see the idea of different "Python
runtimes":

   https://pythoncapi.readthedocs.io/runtimes.html

Replacing macros with function calls is only a first step. It doesn't
solve the problem of borrowed references for example.

Obviously, such change has a cost on performances. Sadly, I didn't run
a benchmark yet. At this point, I mostly care about correctness and
the feasibility of the whole project. I also hope that the new C API
will allow to implement new optimizations which cannot even be
imagined today, because of the backward compatibility. The question is
if the performance balance is positive or not at the all :-)
Hopefully, there is no urgency to take any decision at this point. The
whole project is experimental and can be cancelled anytime.

Victor