[issue40217] The garbage collector doesn't take in account that objects of heap allocated types hold a strong reference to their type

[Petr Viktorin]

Petr Viktorin <encukou at gmail.com> added the comment:

Ha! I think I found the issue in PySide now. It's different, but it's still a CPython issue. It's actually a variant of the "defining class" problem from PEP 573.


It's legal to get the value of a slot, using PyType_GetSlot.
It's reasonable to assume that what you put in the tp_traverse slot is what you'll get out using PyType_GetSlot.
If a type's tp_traverse tries to call its superclass's tp_traverse, you can get an infinite loop.
See the situation below or run attached reproducer (build the module, import it and exit
the interpreter).

Getting one's superclass is a bit tricky when dealing entirely with FromSpec
types, but it is definitely *possible*. There's a lot of assumptions the code
can reasonably make. In my reproducer, I stored the superclass in a C static
variable (which is perfectly valid if the module cleanly refuses to work with
subinterpreters or interpreter reload).


Base:
- real tp_traverse is PyType_FromSpec_tp_traverse
- user_provided_tp_traverse is the base's original

Subclass:
- real tp_traverse is PyType_FromSpec_tp_traverse
- user_provided_tp_traverse is the subclass' original

So when the subclass is traversed:
- subclass real tp_traverse calls the subclass user_provided_tp_traverse
- subclass user_provided_tp_traverse calls PyType_GetSlot(base, tp_traverse)
  - that is, the base's real tp_traverse
- the base's real tp_traverse calls the **SUBCLASS** user_provided_tp_traverse,
  since that's what's recorded in type(self)



Another issue is that `PyType_FromSpec_Alloc`ated types lie about their size:
`tp->tp_basicsize + Py_SIZE(self) * tp->itemsize` is not the actual
allocated amount. This could technically be solved by redefining `__sizeof__`,
but I'm more worried that it's another edge case of a hack, and There will
probably be other edge cases. Since this is API we want to build on, I'd sleep
easier if it's kept clean.


What would be the downsides of reverting and documenting that tp_traverse
must visit Py_TYPE(self)?
It seems that you visit Py_TYPE(self), it could just end up with unbreakable
reference cycles. We're dealing with modules and classes, which are usually
effectively immortal (and if not, the author probably knows what they're doing).
I don't think it would hurt that much.

----------
resolution: fixed -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue40217>
_______________________________________