[issue42294] [C API] Add new C functions with more regular reference counting like PyTuple_GetItemRef()

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

Mark Shannon:
> The old functions aren't going away, so these additional functions provide no real safety. You can't stop C programmers trading away correctness for some perceived performance benefit :(

In my experience, newcomers tend to copy existing more. If slowly, the code base moves towards safer code less error-prone code like Py_NewRef() or Py_SETREF(), slowly, we will avoid a bunch of bugs.


> If we were designing the API from scratch, then this would be a better set of functions. But because the old functions remain, it just means we are making the API larger.

New API VS enhance the existing API. So far, no approach won. I wrote the PEP 620 to enhance the C API and towards a more opaque API, and there is the HPy project which is a new API written correctly from the start. But HPy is not usable yet, and migrating C extensions to HPy will take years.

Also, enhancing the existing API and writing a new API are not exclusive option.

What is the issue of making the C API larger?


> Please don't add macros, use inline functions.

For Py_NewRef(), I used all at once :-) static inline function + function + macro :-)

It's exported as a regular function for the stable ABI, but overriden by a static inline function with a macro. The idea is to allow to use it for developers who cannot use static inline functions (ex: extension modules not written in C).

I chose to redefine functions as static inline functions in the limited C API. If it's an issue, we can consider to only do that in Include/cpython/object.h.


> There seems to be some confusion about borrowed references and stolen references in https://pythoncapi.readthedocs.io/bad_api.html#borrowed-references
"Stealing" a reference is perfectly safe. Returning a "borrowed" reference is not.
>
> So, don't bother with `PyTuple_SetItemRef()`, as `PyTupleSetItem()` is safe.

I'm really annoyed that almost all functions increase the refcount of their arugments, except a bunch of special cases. I would like to move towards a more regular API.

PyTuple_SetItem() is annoying because it steals a reference to the item. Also, it doesn't clear the reference of the previous item, which is also likely to introduce a reference leak.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue42294>
_______________________________________