[issue6128] Consequences of using Py_TPFLAGS_HAVE_GC are incompletely explained

[Christian Aichinger]

Christian Aichinger added the comment:

I concur that this aspect of writing Python types in C/C++ needs better
documentation. For a new extension writer (like me), it is not obvious when
PyObject_GC_Track/PyObject_GC_UnTrack must be called or not. Similarly, it is
not clear from the current documentation how to ensure that memory is allocated
with PyObject_GC_New.

The gcsupport documentation suffers from being written from the perspective
of the GC, instead of the perspective of extension writers. Thus, I've attached
a patch that tries to shed some light on how to correctly write types
implementing cyclic GC support.

I'd appreciate comments/editing suggestions/advice and would welcome the patch
to go into Python in some form. I myself consider the change to be trivial, but
if necessary I can sign the contributor agreement.

--
PS: Informal explanation about what's really going on:

The Python GC code deals with memory allocation, not with initialization in any
way. For most extension types, allocation is handled by tp_alloc which is
called typically called from tp_new (either the default tp_new or your own
tp_new handler).

The default tp_alloc (PyType_GenericAlloc()) looks at tp_flags, and if
Py_TPFLAGS_HAVE_GC is specified, it uses the appropriate GC malloc function and
calls _PyObject_GC_TRACK when finished). Thus, if you allocate your memory via
tp_alloc, and don't override the tp_alloc handler, you've just fulfilled the
GC's constructor requirements.

Similarly, the GC doesn't care about object destruction. It cares about freeing
memory, that is tp_free, NOT tp_dealloc! Again, if you don't intentionally
provide a custom tp_free handler, the default one will take care to obey the GC
rules.

Thus, most extension writers can get away with 4 simple rules:
* Set Py_TPFLAGS_HAVE_GC in tp_flags.
* Do not specify tp_alloc or tp_free.
* Make sure your object actually allocates it's memory via tp_alloc.
  Three possibilities to achieve this:
    + Do not add a tp_new handler, or
    + Call tp_alloc from your tp_new handler, or
    + Call a base class's tp_new from your own tp_new handler
* Implement tp_traverse and tp_clear. They are usually straight-forward.

And if you really do want a custom allocator, you're screwed anyway, as the
last part of my patch hints at. Custom memory management is just not compatible with the GC at this point.

----------
keywords: +patch
nosy: +Greek0
versions: +Python 3.5
Added file: http://bugs.python.org/file33497/gcsupport-doc.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue6128>
_______________________________________