[Cython] Should we start using the internal CPython APIs?

[Stefan Behnel]

da-woods schrieb am 04.11.23 um 14:45:
> I'm a bit late in replying to this but here are some unordered thoughts.
> 
> * I'm fairly relaxed about using `Py_BUILD_CORE` if useful - I think we 
> mostly do have good fallback paths for most things so can adapt quickly 
> when stuff changes.

I'm not entirely relaxed about it, but I agree that the fallbacks should 
usually make it easy to keep things working also after larger changes in 
CPython.


> * CYTHON_USE_CPYTHON_CORE_DETAILS sounds reasonable, but it's yet another 
> variation to test.

True.


> * I wonder if fixing up the limited API implementation should be higher 
> priority than creating a third level been "full" and "limited API".

I think there's potential for all three. Basically modes "aggressively 
fast", "highly compatible" and "version independent". The latter is what 
the Stable ABI together with the Limited API should give you.


> * I recall we were planning to ditch c89 as a strict requirement after 3.0? 
> Incompatibility with C++ might be more of an issue though.

Yes. C++ is not an issue for CPython, so their internal header files are 
not tested with C++ at all. That's the highest potential for breakage, if 
we accept to generate C99 from Cython 3.1 onwards.

We should make sure that we use "-std=c89" in at least one Cython 3.0 test 
setup, BTW.


> * Even so, if there's a good way of turning it off then we could say: "if 
> you want strict c89 support then you can't use 
> CYTHON_USE_CPYTHON_CORE_DETAILS" and people would always have options.

That could be part of it, yes.


> * Waiting and seeing may be a good option for now.

I agree. This still seems best for now, especially given the amount of 
recent changes in the C-API. Let's wait for those to settle down, at least.

Thanks everyone for your opinions and comments!

Stefan