frozendict: an experiment

[Marco Sulla]

On Fri, 17 Jul 2020 at 04:13, Inada Naoki <songofacandy at gmail.com> wrote:
> > 3. many python internals uses a mapping proxy to a dict, to avoid its
> > modification. A frozendict can be used instead.
>
> Are they used frequently in performance critical path?
> Could you point some concrete examples?

I searched a little in CPython code, and it seems that MappingProxy is
used in a number of critical points.

In re:
./Modules/_sre.c:    return PyDictProxy_New(self->groupindex);

in mutiprocessing:
./Lib/multiprocessing/managers.py:DictProxy = MakeProxyType('DictProxy', (
./Lib/multiprocessing/managers.py:DictProxy._method_to_typeid_ = {
./Lib/multiprocessing/managers.py:SyncManager.register('dict', dict, DictProxy)

In functools:
./Lib/functools.py:    wrapper.registry = types.MappingProxyType(registry)

In enum:
./Lib/enum.py:        return MappingProxyType(cls._member_map_)

I suppose the more crucial is _sre, since it's used extensively in
CPython. groupindex is used by a large number of _sre functions.
Note: I'm not sure that mappingproxyobject is much slower than dict,
as types.MappyingProxyType.

On Fri, 17 Jul 2020 at 04:13, Inada Naoki <songofacandy at gmail.com> wrote:
> I am not sure tuple is "internined" or just "merged". (I don't know precise
> definition of the "interning").
>
> Tuples are merged while compiling.
>
> ```
>     for a in ["foo", "bar"]:  # compiler convert this list to tuple
>         ...
>     for b in ("foo", "bar"):  # and compiler merge same tuple
>         ...
> ```
>
> But when frozendicts are merged?
> I think there is a very little chance.

frozendicts could be used for kwargs:

f(a=1, b=2)
# some code
f(a=1, b=2)

For what I know, CPython uses PyDictObject for kwargs. Since dicts are
mutable, it's a problem to cache them properly.

On Fri, 17 Jul 2020 at 04:13, Inada Naoki <songofacandy at gmail.com> wrote:
> I'm OK to all combined dict for frozen dict.  But I think key-sharing is still
> interesting optimization for frozen dict. And supporting key-sharing dict
> is much easier for frozendict than for mutable dict.

Yes, I think the same.

On Fri, 17 Jul 2020 at 04:13, Inada Naoki <songofacandy at gmail.com> wrote:
> Then, there is no reason to not support the view for frozendict?

I didn't say to not support views... I said that "real objects", that
implement the dictview API, could be returned by frozendict.keys()
etc.