[Python-Dev] Inconsistency of PyModule_AddObject()

[Serhiy Storchaka]

There are three functions (or at least three documented functions) in C 
API that "steals" references: PyList_SetItem(), PyTuple_SetItem() and 
PyModule_AddObject(). The first two "steals" references even on failure, 
and this is well known behaviour. But PyModule_AddObject() "steals" a 
reference only on success. There is nothing in the documentation that 
points on this. Most usages of PyModule_AddObject() in the stdlib don't 
decref the reference to the value on PyModule_AddObject() failure. The 
only exceptions are in _json, _io, and _tkinter modules. In many cases, 
including examples in the documentation, the successfulness of 
PyModule_AddObject() is not checked either, but this is different issue.

We can just fix the documentation but adding a note that 
PyModule_AddObject() doesn't steal a reference on failure. And add 
explicit decrefs after PyModule_AddObject() in hundreds of places in the 
code.

But I think it would be better to "fix" PyModule_AddObject() by making 
it decrefing a reference on failure as expected by most developers. But 
this is dangerous change, because if the author of third-party code read 
not only the documentation, but CPython code, and added explicit decref 
on PyModule_AddObject() failure, we will get a double decrefing.

I think that we can resolve this issue by following steps:

1. Add a new function PyModule_AddObject2(), that steals a reference 
even on failure.

2. Introduce a special macro like PY_SSIZE_T_CLEAN (any suggestions 
about a name?). If it is defined, define PyModule_AddObject as 
PyModule_AddObject2. Define this macro before including Python.h in all 
CPython modules except _json, _io, and _tkinter.

3. Make old PyModule_AddObject to emit a warning about possible leak and 
a suggestion to define above macro.