[Python-checkins] GH-95245: Document use of `MANAGED` flags instead of offsets. (GH-96044)
markshannon
webhook-mailer at python.org
Tue Aug 30 11:26:21 EDT 2022
https://github.com/python/cpython/commit/0f733fffe8f4caaac3ce1b5306af86b42fb0c7fa
commit: 0f733fffe8f4caaac3ce1b5306af86b42fb0c7fa
branch: main
author: Mark Shannon <mark at hotpy.org>
committer: markshannon <mark at hotpy.org>
date: 2022-08-30T16:26:08+01:00
summary:
GH-95245: Document use of `MANAGED` flags instead of offsets. (GH-96044)
files:
M Doc/c-api/structures.rst
M Doc/c-api/type.rst
M Doc/c-api/typeobj.rst
M Doc/extending/newtypes.rst
diff --git a/Doc/c-api/structures.rst b/Doc/c-api/structures.rst
index 86d4536f8d28..f1eb09bb5691 100644
--- a/Doc/c-api/structures.rst
+++ b/Doc/c-api/structures.rst
@@ -469,19 +469,21 @@ Accessing attributes of extension types
.. _pymemberdef-offsets:
Heap allocated types (created using :c:func:`PyType_FromSpec` or similar),
- ``PyMemberDef`` may contain definitions for the special members
- ``__dictoffset__``, ``__weaklistoffset__`` and ``__vectorcalloffset__``,
- corresponding to
- :c:member:`~PyTypeObject.tp_dictoffset`,
- :c:member:`~PyTypeObject.tp_weaklistoffset` and
+ ``PyMemberDef`` may contain definitions for the special member
+ ``__vectorcalloffset__``, corresponding to
:c:member:`~PyTypeObject.tp_vectorcall_offset` in type objects.
These must be defined with ``T_PYSSIZET`` and ``READONLY``, for example::
static PyMemberDef spam_type_members[] = {
- {"__dictoffset__", T_PYSSIZET, offsetof(Spam_object, dict), READONLY},
+ {"__vectorcalloffset__", T_PYSSIZET, offsetof(Spam_object, vectorcall), READONLY},
{NULL} /* Sentinel */
};
+ The legacy offsets :c:member:`~PyTypeObject.tp_dictoffset` and
+ :c:member:`~PyTypeObject.tp_weaklistoffset` are still supported, but extensions are
+ strongly encouraged to use ``Py_TPFLAGS_MANAGED_DICT`` and
+ ``Py_TPFLAGS_MANAGED_WEAKREF`` instead.
+
.. c:function:: PyObject* PyMember_GetOne(const char *obj_addr, struct PyMemberDef *m)
diff --git a/Doc/c-api/type.rst b/Doc/c-api/type.rst
index aa77c285e3b8..deb5502a4dff 100644
--- a/Doc/c-api/type.rst
+++ b/Doc/c-api/type.rst
@@ -327,9 +327,9 @@ The following functions and structs are used to create
* :c:member:`~PyTypeObject.tp_weaklist`
* :c:member:`~PyTypeObject.tp_vectorcall`
* :c:member:`~PyTypeObject.tp_weaklistoffset`
- (see :ref:`PyMemberDef <pymemberdef-offsets>`)
+ (use :const:`Py_TPFLAGS_MANAGED_WEAKREF` instead)
* :c:member:`~PyTypeObject.tp_dictoffset`
- (see :ref:`PyMemberDef <pymemberdef-offsets>`)
+ (use :const:`Py_TPFLAGS_MANAGED_DICT` instead)
* :c:member:`~PyTypeObject.tp_vectorcall_offset`
(see :ref:`PyMemberDef <pymemberdef-offsets>`)
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index 20cdffed110c..2439f7c41b56 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -96,7 +96,7 @@ Quick Reference
| | | __gt__, | | | | |
| | | __ge__ | | | | |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
- | :c:member:`~PyTypeObject.tp_weaklistoffset` | :c:type:`Py_ssize_t` | | | X | | ? |
+ | (:c:member:`~PyTypeObject.tp_weaklistoffset`) | :c:type:`Py_ssize_t` | | | X | | ? |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
| :c:member:`~PyTypeObject.tp_iter` | :c:type:`getiterfunc` | __iter__ | | | | X |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
@@ -117,7 +117,7 @@ Quick Reference
| :c:member:`~PyTypeObject.tp_descr_set` | :c:type:`descrsetfunc` | __set__, | | | | X |
| | | __delete__ | | | | |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
- | :c:member:`~PyTypeObject.tp_dictoffset` | :c:type:`Py_ssize_t` | | | X | | ? |
+ | (:c:member:`~PyTypeObject.tp_dictoffset`) | :c:type:`Py_ssize_t` | | | X | | ? |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
| :c:member:`~PyTypeObject.tp_init` | :c:type:`initproc` | __init__ | X | X | | X |
+------------------------------------------------+-----------------------------------+-------------------+---+---+---+---+
@@ -1018,7 +1018,6 @@ and :c:type:`PyType_Type` effectively act as defaults.)
:const:`Py_TPFLAGS_HAVE_GC` flag bit is clear in the subtype and the
:c:member:`~PyTypeObject.tp_traverse` and :c:member:`~PyTypeObject.tp_clear` fields in the subtype exist and have
``NULL`` values.
-
.. XXX are most flag bits *really* inherited individually?
**Default:**
@@ -1135,6 +1134,33 @@ and :c:type:`PyType_Type` effectively act as defaults.)
:const:`Py_TPFLAGS_IMMUTABLETYPE` flag set. For extension types, it is
inherited whenever :c:member:`~PyTypeObject.tp_descr_get` is inherited.
+ .. data:: Py_TPFLAGS_MANAGED_DICT
+
+ This bit indicates that instances of the class have a ``__dict___``
+ attribute, and that the space for the dictionary is managed by the VM.
+
+ If this flag is set, :const:`Py_TPFLAGS_HAVE_GC` should also be set.
+
+ .. versionadded:: 3.12
+
+ **Inheritance:**
+
+ This flag is inherited unless the
+ :c:member:`~PyTypeObject.tp_dictoffset` field is set in a superclass.
+
+
+ .. data:: Py_TPFLAGS_MANAGED_WEAKREF
+
+ This bit indicates that instances of the class should be weakly
+ referenceable.
+
+ .. versionadded:: 3.12
+
+ **Inheritance:**
+
+ This flag is inherited unless the
+ :c:member:`~PyTypeObject.tp_weaklistoffset` field is set in a superclass.
+
.. XXX Document more flags here?
@@ -1487,6 +1513,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)
.. c:member:: Py_ssize_t PyTypeObject.tp_weaklistoffset
+ While this field is still supported, :const:`Py_TPFLAGS_MANAGED_WEAKREF`
+ should be used instead, if at all possible.
+
If the instances of this type are weakly referenceable, this field is greater
than zero and contains the offset in the instance structure of the weak
reference list head (ignoring the GC header, if present); this offset is used by
@@ -1497,6 +1526,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)
Do not confuse this field with :c:member:`~PyTypeObject.tp_weaklist`; that is the list head for
weak references to the type object itself.
+ It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and
+ :c:member:`~PyTypeObject.tp_weaklist`.
+
**Inheritance:**
This field is inherited by subtypes, but see the rules listed below. A subtype
@@ -1504,19 +1536,12 @@ and :c:type:`PyType_Type` effectively act as defaults.)
reference list head than the base type. Since the list head is always found via
:c:member:`~PyTypeObject.tp_weaklistoffset`, this should not be a problem.
- When a type defined by a class statement has no :attr:`~object.__slots__` declaration,
- and none of its base types are weakly referenceable, the type is made weakly
- referenceable by adding a weak reference list head slot to the instance layout
- and setting the :c:member:`~PyTypeObject.tp_weaklistoffset` of that slot's offset.
-
- When a type's :attr:`__slots__` declaration contains a slot named
- :attr:`__weakref__`, that slot becomes the weak reference list head for
- instances of the type, and the slot's offset is stored in the type's
- :c:member:`~PyTypeObject.tp_weaklistoffset`.
+ **Default:**
- When a type's :attr:`__slots__` declaration does not contain a slot named
- :attr:`__weakref__`, the type inherits its :c:member:`~PyTypeObject.tp_weaklistoffset` from its
- base type.
+ If the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit is set in the
+ :c:member:`~PyTypeObject.tp_dict` field, then
+ :c:member:`~PyTypeObject.tp_weaklistoffset` will be set to a negative value,
+ to indicate that it is unsafe to use this field.
.. c:member:: getiterfunc PyTypeObject.tp_iter
@@ -1695,6 +1720,9 @@ and :c:type:`PyType_Type` effectively act as defaults.)
.. c:member:: Py_ssize_t PyTypeObject.tp_dictoffset
+ While this field is still supported, :const:`Py_TPFLAGS_MANAGED_DICT` should be
+ used instead, if at all possible.
+
If the instances of this type have a dictionary containing instance variables,
this field is non-zero and contains the offset in the instances of the type of
the instance variable dictionary; this offset is used by
@@ -1703,17 +1731,7 @@ and :c:type:`PyType_Type` effectively act as defaults.)
Do not confuse this field with :c:member:`~PyTypeObject.tp_dict`; that is the dictionary for
attributes of the type object itself.
- If the value of this field is greater than zero, it specifies the offset from
- the start of the instance structure. If the value is less than zero, it
- specifies the offset from the *end* of the instance structure. A negative
- offset is more expensive to use, and should only be used when the instance
- structure contains a variable-length part. This is used for example to add an
- instance variable dictionary to subtypes of :class:`str` or :class:`tuple`. Note
- that the :c:member:`~PyTypeObject.tp_basicsize` field should account for the dictionary added to
- the end in that case, even though the dictionary is not included in the basic
- object layout. On a system with a pointer size of 4 bytes,
- :c:member:`~PyTypeObject.tp_dictoffset` should be set to ``-4`` to indicate that the dictionary is
- at the very end of the structure.
+ The value specifies the offset of the dictionary from the start of the instance structure.
The :c:member:`~PyTypeObject.tp_dictoffset` should be regarded as write-only.
To get the pointer to the dictionary call :c:func:`PyObject_GenericGetDict`.
@@ -1721,30 +1739,26 @@ and :c:type:`PyType_Type` effectively act as defaults.)
dictionary, so it is may be more efficient to call :c:func:`PyObject_GetAttr`
when accessing an attribute on the object.
- **Inheritance:**
-
- This field is inherited by subtypes, but see the rules listed below. A subtype
- may override this offset; this means that the subtype instances store the
- dictionary at a difference offset than the base type. Since the dictionary is
- always found via :c:member:`~PyTypeObject.tp_dictoffset`, this should not be a problem.
+ It is an error to set both the :const:`Py_TPFLAGS_MANAGED_WEAKREF` bit and
+ :c:member:`~PyTypeObject.tp_dictoffset`.
- When a type defined by a class statement has no :attr:`~object.__slots__` declaration,
- and none of its base types has an instance variable dictionary, a dictionary
- slot is added to the instance layout and the :c:member:`~PyTypeObject.tp_dictoffset` is set to
- that slot's offset.
-
- When a type defined by a class statement has a :attr:`__slots__` declaration,
- the type inherits its :c:member:`~PyTypeObject.tp_dictoffset` from its base type.
+ **Inheritance:**
- (Adding a slot named :attr:`~object.__dict__` to the :attr:`__slots__` declaration does
- not have the expected effect, it just causes confusion. Maybe this should be
- added as a feature just like :attr:`__weakref__` though.)
+ This field is inherited by subtypes. A subtype should not override this offset;
+ doing so could be unsafe, if C code tries to access the dictionary at the
+ previous offset.
+ To properly support inheritance, use :const:`Py_TPFLAGS_MANAGED_DICT`.
**Default:**
This slot has no default. For :ref:`static types <static-types>`, if the
field is ``NULL`` then no :attr:`__dict__` gets created for instances.
+ If the :const:`Py_TPFLAGS_MANAGED_DICT` bit is set in the
+ :c:member:`~PyTypeObject.tp_dict` field, then
+ :c:member:`~PyTypeObject.tp_dictoffset` will be set to ``-1``, to indicate
+ that it is unsafe to use this field.
+
.. c:member:: initproc PyTypeObject.tp_init
@@ -2663,8 +2677,6 @@ A type that supports weakrefs, instance dicts, and hashing::
typedef struct {
PyObject_HEAD
const char *data;
- PyObject *inst_dict;
- PyObject *weakreflist;
} MyObject;
static PyTypeObject MyObject_Type = {
@@ -2672,9 +2684,9 @@ A type that supports weakrefs, instance dicts, and hashing::
.tp_name = "mymod.MyObject",
.tp_basicsize = sizeof(MyObject),
.tp_doc = PyDoc_STR("My objects"),
- .tp_weaklistoffset = offsetof(MyObject, weakreflist),
- .tp_dictoffset = offsetof(MyObject, inst_dict),
- .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE | Py_TPFLAGS_HAVE_GC,
+ .tp_flags = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
+ Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_MANAGED_DICT |
+ Py_TPFLAGS_MANAGED_WEAKREF,
.tp_new = myobj_new,
.tp_traverse = (traverseproc)myobj_traverse,
.tp_clear = (inquiry)myobj_clear,
diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
index c7c434e58bf0..b797dc2817c8 100644
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -570,43 +570,28 @@ performance-critical objects (such as numbers).
.. seealso::
Documentation for the :mod:`weakref` module.
-For an object to be weakly referencable, the extension type must do two things:
+For an object to be weakly referencable, the extension type must set the
+``Py_TPFLAGS_MANAGED_WEAKREF`` bit of the :c:member:`~PyTypeObject.tp_flags`
+field. The legacy :c:member:`~PyTypeObject.tp_weaklistoffset` field should
+be left as zero.
-#. Include a :c:type:`PyObject\*` field in the C object structure dedicated to
- the weak reference mechanism. The object's constructor should leave it
- ``NULL`` (which is automatic when using the default
- :c:member:`~PyTypeObject.tp_alloc`).
-
-#. Set the :c:member:`~PyTypeObject.tp_weaklistoffset` type member
- to the offset of the aforementioned field in the C object structure,
- so that the interpreter knows how to access and modify that field.
-
-Concretely, here is how a trivial object structure would be augmented
-with the required field::
-
- typedef struct {
- PyObject_HEAD
- PyObject *weakreflist; /* List of weak references */
- } TrivialObject;
-
-And the corresponding member in the statically declared type object::
+Concretely, here is how the statically declared type object would look::
static PyTypeObject TrivialType = {
PyVarObject_HEAD_INIT(NULL, 0)
/* ... other members omitted for brevity ... */
- .tp_weaklistoffset = offsetof(TrivialObject, weakreflist),
+ .tp_flags = Py_TPFLAGS_MANAGED_WEAKREF | ...,
};
+
The only further addition is that ``tp_dealloc`` needs to clear any weak
-references (by calling :c:func:`PyObject_ClearWeakRefs`) if the field is
-non-``NULL``::
+references (by calling :c:func:`PyObject_ClearWeakRefs`)::
static void
Trivial_dealloc(TrivialObject *self)
{
/* Clear weakrefs first before calling any destructors */
- if (self->weakreflist != NULL)
- PyObject_ClearWeakRefs((PyObject *) self);
+ PyObject_ClearWeakRefs((PyObject *) self);
/* ... remainder of destruction code omitted for brevity ... */
Py_TYPE(self)->tp_free((PyObject *) self);
}
More information about the Python-checkins
mailing list