[New-bugs-announce] [issue45233] Allow split key dictionaries with values owned by other objects.

[Mark Shannon]

New submission from Mark Shannon <mark at hotpy.org>:

Currently, if a dictionary is split, then the dictionary owns the memory for the values. Unless the values is the unique empty-values array.

In order to support lazily created dictionaries for objects (see https://github.com/faster-cpython/ideas/issues/72#issuecomment-886796360), we need to allow shared keys dicts that do not own the memory of the values.

I propose the following changes to the internals of dict objects.
Add 4 flag bits (these can go in the low bits of the version tag)
2 bit for ownership of values, the other 2 bits for the stride of the values (1 or 3). All dictionaries would then have a non-NULL values pointer.

The value of index `ix` would be always located at `dict->ma_values[ix*stride]`

The two ownership bits indicate whether the dictionary owns the references and whether it owns the memory.
When a dictionary is freed, the items in the values array would be decref'd if the references are owned.
The values array would be freed if the memory is owned.

I don't think it is possible to own the memory, but not the references.

Examples:

A combined dict. Stride = 3, owns_refs = 1, owns_mem = 0.
A split keys dict. Stride = 1, owns_refs = 1, owns_mem = 1.
Empty dict (split). Stride = 1, owns_refs = 0, owns_mem = 0.

Dictionary with values embedded in object (https://github.com/faster-cpython/ideas/issues/72#issuecomment-886796360, second diagram). Stride = 1, owns_refs = 0, owns_mem = 0.

----------
assignee: Mark.Shannon
components: Interpreter Core
messages: 402047
nosy: Mark.Shannon, methane
priority: normal
severity: normal
status: open
title: Allow split key dictionaries with values owned by other objects.

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue45233>
_______________________________________