[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

This bug is quite complex. Let me try to explain it more simply.

* Create an object A
* Create a weak reference to A with a callback CB
* A and CB are part of a reference cycle
* Removing the last references to A and CB are not enough to destroy them
* Trigger a garbage collection to break the cycle
* A and CB are seen as unreachable by the GC
* The GC "clears" CB which makes CB inconsistent (tp_clear)
* A is deleted which calls CB
* Crash on calling inconsistent CB

--

In the case of FreeIPA:

* A is a _cffi_backend.CTypeDescr instance
* CB is the remove() function defined in weakref.WeakValueDictionary constructor

The FreeIPA reference cycle is quite complex:

remove()
-> remove().__closure__ (tuple)
-> WeakValueDictionary.data (dict) = remove().__closure__[0]
-> weakref.KeyedRef
-> (<str 1>, <tuple 2>)
-> (<CTypeDescrObject 3>,) = <tuple 2>
-> <CTypeDescrObject 3>
-> {'C_Initialize': ..., 'C_CreateObject': ..., 'C_Finalize': ...} (dict)
-> <_cffi_backend.CField 4>
-> <_cffi_backend.CTypeDescr 5>
-> remove() = callback of a weak reference to <_cffi_backend.CTypeDescr 5>

<_cffi_backend.CField 4> is not seen as unreachable by the GC because CField_Type.tp_traverse is not implemented (and CField_Type doesn't use Py_TPFLAGS_HAVE_GC flag).

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38006>
_______________________________________