[New-bugs-announce] [issue45490] [meta][C API] Avoid C macro pitfalls and usage of static inline functions

[STINNER Victor]

New submission from STINNER Victor <vstinner at python.org>:

C macros are really cool and useful, but there are a bunch of pitfalls which are better to avoid:
https://gcc.gnu.org/onlinedocs/cpp/Macro-Pitfalls.html

Some macros of the Python C API have been converted to static inline functions over the last years. It went smoothly, I am not aware of any major issue with these conversions.

This meta issue tracks other issues related to macros and static inline functions.


=== Return void ===

One issue is that some macros are treated as an expression and can be reused, whereas it was not intended. For example PyList_SET_ITEM() was implemented as (simplified code):

  #define PyList_SET_ITEM(op, i, v) (op->ob_item[i] = v)

This expression has a value! Two projects used this value by mistake, like:

  "if (obj == NULL || PyList_SET_ITEM (l, i, obj) < 0)"

PyList_SET_ITEM() was fixed by casting the expression to void:

  #define PyList_SET_ITEM(op, i, v) ((void)(op->ob_item[i] = v))

=> bpo-30459



=== Abuse macros as an l-value ===

The Py_TYPE() macro could be used to assign a value: "Py_TYPE(obj) = new_type".

The macro was defined as:

  #define Py_TYPE(ob) (ob->ob_type)

It was converted to a static inline function to disallow using it as an l-value and a new Py_SET_TYPE(op, new_type) function was added. These changes give more freedom to other Python implementations to implement "PyObject" and Py_SET_TYPE().

=> bpo-45476 "[C API] Disallow using PyFloat_AS_DOUBLE() as l-value"
=> bpo-39573 PyObject Py_TYPE/Py_SET_TYPE


=== C API: Macros and embedded Python ===

Sadly, only symbols exported by libpython are accessible to other programming languages embedding Python. Macros of the Python C API are simply not available to them. Projects embedding Python have to hardcode constants and copy macros to their own language, with the risk of being outdated when Python macros are updated.

Even some projects written in C cannot use macros, because they only use libpython symbols. The vim text editor embeds Python this way.

Also, macros are simply excluded from the Python stable ABI (PEP 384).


=== Performance of static inline functions ===

In bpo-45116, it seems like _PyEval_EvalFrameDefault() reached Visual Studio thresholds and some static inline functions are no longer inlined (Py_INCREF/Py_DECREF).

I also noticed that when Python is built in debug mode in Visual Studio, static inline functions are not inlined. Well, the compiler is free to not inline in debug mode. I guess that GCC and clang also skip inlining using -Og and/or -O0 optimization levels. Using __forceinline and __attribute__((always_inline)) on static inline functions (Py_INCREF, Py_TYPE) for debug builds was discussed in bpo-45094, but the idea was rejected.

On the other side, sometimes it's better to *disable* inlining on purpose to reduce the stack memory consumption, using the Py_NO_INLINE macro. See recent measurements of the stack memory usage:
https://bugs.python.org/issue45439#msg403768

In GH-28893, I noticed that converting a static inline function (PyObject_CallOneArg) to a regular function made it faster. I am not really sure, more benchmarks should be run to really what's going on.


=== Advantages of static inline functions ===

* It's possible to put a breakpoint on a static inline functions.

* Debuggers and profilers are able to get the static inline function names from the machine line, even with inline functions.

* Parameters and the return value have well defined types.

* Variables have a local scope.

* There is no risk of evaluating an expression multiple times.

* Regular C code. No need to use "\" character to multi-line statement. No need for "do { ... } while (0)" and other quicks to workaround preprocessor pitfalls. No abuse of (((parenthesis))).

----------
components: C API
messages: 404038
nosy: vstinner
priority: normal
severity: normal
status: open
title: [meta][C API] Avoid C macro pitfalls and usage of static inline functions
versions: Python 3.11

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45490>
_______________________________________