[New-bugs-announce] [issue44772] Regression in memory use of instances due to dictionary ordering

[Mark Shannon]

New submission from Mark Shannon <mark at hotpy.org>:

class C:
    def __init__(self, cond):
        if cond:
            self.a = 1
        self.b = 2

c1 = C(True)
c2 = C(False)

   
In Python 3.5, the dictionary keys are shared
---------------------------------------------

>>> sys.getsizeof(c2)
56
>>> sys.getsizeof(c1.__dict__)
96
>>> sys.getsizeof(c2.__dict__)
96

In Python 3.9, the keys are not shared
--------------------------------------

>>> sys.getsizeof(c2)
48
>>> sys.getsizeof(c1.__dict__)
272
>>> sys.getsizeof(c2.__dict__)
232

This represents an increase of memory use for c1 of 110%. (48+272)/(56+96) == 2.1

With compact object layout (https://github.com/faster-cpython/ideas/issues/69), 
any failure to share keys will result in a tripling of memory use per object.

It is not an uncommon pattern for some attributes to initialized lazily,
which causes also prevents key-sharing.

Not only does it increase memory use, it prevents optimizations that attempt to specialize attribute access for instances of a class, as different instances will have different layouts.

The purpose of compact dicts was to save memory, but in this case we are using a lot more memory by preventing PEP 412 from working.

----------
keywords: 3.9regression
messages: 398475
nosy: Mark.Shannon
priority: normal
severity: normal
status: open
title: Regression in memory use of instances due to dictionary ordering

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue44772>
_______________________________________