[Python-Dev] Inconsistency of PyModule_AddObject()

[Serhiy Storchaka]

On 27.04.16 10:14, Serhiy Storchaka wrote:
> There are three functions (or at least three documented functions) in C
> API that "steals" references: PyList_SetItem(), PyTuple_SetItem() and
> PyModule_AddObject(). The first two "steals" references even on failure,
> and this is well known behaviour. But PyModule_AddObject() "steals" a
> reference only on success. There is nothing in the documentation that
> points on this. Most usages of PyModule_AddObject() in the stdlib don't
> decref the reference to the value on PyModule_AddObject() failure. The
> only exceptions are in _json, _io, and _tkinter modules. In many cases,
> including examples in the documentation, the successfulness of
> PyModule_AddObject() is not checked either, but this is different issue.
>
> We can just fix the documentation but adding a note that
> PyModule_AddObject() doesn't steal a reference on failure. And add
> explicit decrefs after PyModule_AddObject() in hundreds of places in the
> code.
>
> But I think it would be better to "fix" PyModule_AddObject() by making
> it decrefing a reference on failure as expected by most developers. But
> this is dangerous change, because if the author of third-party code read
> not only the documentation, but CPython code, and added explicit decref
> on PyModule_AddObject() failure, we will get a double decrefing.
>
> I think that we can resolve this issue by following steps:
>
> 1. Add a new function PyModule_AddObject2(), that steals a reference
> even on failure.
>
> 2. Introduce a special macro like PY_SSIZE_T_CLEAN (any suggestions
> about a name?). If it is defined, define PyModule_AddObject as
> PyModule_AddObject2. Define this macro before including Python.h in all
> CPython modules except _json, _io, and _tkinter.
>
> 3. Make old PyModule_AddObject to emit a warning about possible leak and
> a suggestion to define above macro.

Opened an issue: http://bugs.python.org/issue26871 .

Provided patch introduces new macros PY_MODULE_ADDOBJECT_CLEAN that 
controls the behavior of PyModule_AddObject() as PY_SSIZE_T_CLEAN 
controls the behavior of PyArg_Parse* functions. If the macro is defined 
before including "Python.h", PyModule_AddObject() steals a reference 
unconditionally.  Otherwise it steals a reference only on success, and 
the caller is responsible for decref'ing it on error (current behavior).