[issue38006] Crash in remove() weak reference callback of weakref.WeakValueDictionary at Python exit

[STINNER Victor]

STINNER Victor <vstinner at python.org> added the comment:

I investigated the FreeIPA crash.

* Python 3.8 behaves differently because func_clear() has been implemented (bpo-33418, commit 3c452404ae178b742967589a0bb4a5ec768d76e0)

* The bug is a crash on a function call (_PyFunction_Vectorcall) because the function has been cleared (by func_clear), but there was still a weak reference using this function as its callback.

* Note: the function is called *during* it's being cleared by func_clear().

* The GC has a workaround for weak references part of "unreachable" objects, but its handle_weakrefs() function doesn't work because CFFI CField_Type type doesn't implement tp_traverse.

--

PR 15641 just hides the real bug.

One issue is that CFFI doesn't implement correctly the GC protocol. If an object contains another object, its type must:

* Have Py_TPFLAGS_HAVE_GC flag
* Implement tp_traverse
* Use PyObject_GC_Malloc() to allocate an object
* Call PyObject_GC_Track() on created object

Another issue is that the GC doesn't prevent the crash. Would it be possible to prevent the crash without changing the behavior (ex: still call weakref callbacks)?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38006>
_______________________________________