[issue515074] Extended storage in new-style classes

[Amaury Forgeot d'Arc]

Amaury Forgeot d'Arc added the comment:

Most concrete variable-sized types don't use tp_basicsize to know where the data lives. For example, PyBytesObject has a "char ob_sval[1];" member, and PyBytes_AsString() looks at this fixed offset.
For this reason, additional data cannot be stored before the variable-length section, only after.

It is possible to store data after the variable section, and CPython does it already for the __dict__ slot of user-defined classes::
    >>> bytes.__basicsize__
    33
    >>> class B(bytes): pass
    ...
    >>> B.__basicsize__
    41
    >>> B.__dictoffset__
    -8

Note that tp_basicsize was increased by the size of the additional data (here a PyObject*).
To access your data, the logic is something like::
    tp->tp_basicsize + (obj->ob_size * tp->tp_itemsize) - sizeof(MyData)
The function _PyObject_GetDictPtr() has similar code, and already aligns to the pointer size.

Of course, at the end of one object you cannot have *both* a __dict__ slot and custom data.
The custom type needs to either disallow subclassing, or provide a (negative) tp_dictoffset.
Also, some care may be required if the base class uses a custom allocator.

Overall, I think it's a bit involved, but doable.
I close the issue as "works for me", someone can reopen if there is something wrong in the base types.

----------
nosy: +amaury.forgeotdarc
resolution:  -> works for me
status: open -> closed

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue515074>
_______________________________________