Cpython: when to incref before insertdict

[Inada Naoki]

On Sat, Mar 5, 2022 at 11:22 PM Marco Sulla
<Marco.Sulla.Python at gmail.com> wrote:
>
> I noticed that some functions inside dictobject.c that call insertdict
> or PyDict_SetItem do an incref of key and value before the call, and a
> decref after it. An example is dict_merge.

First of all, insertdict and PyDict is totally different about
reference ownership handling.

* PyDict_SetItem borrows reference of key and value from the caller as
usual Python/C APIs. And it INCREF them before calling the
insertdict().
* insertdict() takes the reference from its caller. In other words,
insertdict() moves the owner of reference from its caller to the dict.

merge_dict is very special and complex case.
I assume you are talking about this part.
https://github.com/python/cpython/blob/6927632492cbad86a250aa006c1847e03b03e70b/Objects/dictobject.c#L2885-L2912

In general, when reference is borrowed from a caller, the reference is
available during the API.
But merge_dict borrows reference of key/value from other dict, not caller.
So dict_merge must have strong reference of key/value by INCREF before
calling any APIs (e.g. _PyDict_Contains_KnownHash).
That's why dict_merge calls INCREF key/value **twice** before calling
insertdict, and DECREF key/value **once** after it.

> Other functions, such as
> _PyDict_FromKeys, don't do an incref before.
>

Again, insertdict takes the reference. So _PyDict_FromKeys() **does**
INCREF before calling insertdict, when key/value is borrowed
reference.
https://github.com/python/cpython/blob/6927632492cbad86a250aa006c1847e03b03e70b/Objects/dictobject.c#L2287-L2290
https://github.com/python/cpython/blob/6927632492cbad86a250aa006c1847e03b03e70b/Objects/dictobject.c#L2309-L2311

On the other hand, slow path uses PyIter_Next() which returns strong
reference. So no need to INCREF it.
Additionally, the slow path uses PyDict_SetItem(), not insertdict().
PyDict_SetItem() does INCREF key/value for insertdict.
So the slow path need to DECREF(key).
https://github.com/python/cpython/blob/6927632492cbad86a250aa006c1847e03b03e70b/Objects/dictobject.c#L2327-L2329

This is complete guide why/when INCREF/DECREF key/value.
-- 
Inada Naoki  <songofacandy at gmail.com>