[Python-Dev] A new dictionary implementation

[Antoine Pitrou]

On Wed, 1 Feb 2012 09:50:55 -0800
Guido van Rossum <guido at python.org> wrote:
> On Wed, Feb 1, 2012 at 9:13 AM, Hans Mulder <hansmu at xs4all.nl> wrote:
> > On 30/01/12 00:30:14, Steven D'Aprano wrote:
> >>
> >> Mark Shannon wrote:
> >>>
> >>> Antoine Pitrou wrote:
> >
> > [......]
> >
> >>> Antoine is right. It is a reorganisation of the dict, plus a couple of
> >>> changes to typeobject.c and object.c to ensure that instance
> >>> dictionaries do indeed share keys arrays.
> >>
> >>
> >>
> >> I don't quite follow how that could work.
> >>
> >> If I have this:
> >>
> >> class C:
> >> pass
> >>
> >> a = C()
> >> b = C()
> >>
> >> a.spam = 1
> >> b.ham = 2
> >>
> >>
> >> how can a.__dict__ and b.__dict__ share key arrays? I've tried reading
> >> the source, but I'm afraid I don't understand it well enough to make
> >> sense of it.
> >
> >
> > They can't.
> >
> > But then, your class is atypical.  Usually, classes initialize all the
> > attributes of their instances in the __init__ method, perhaps like so:
> >
> > class D:
> >    def __init__(self, ham=None, spam=None):
> >        self.ham = ham
> >        self.spam = spam
> >
> > As long as you follow the common practice of not adding any attributes
> > after the object has been initialized, your instances can share their
> > keys array.  Mark's patch will do that.
> >
> > You'll still be allowed to have different attributes per instance, but
> > if you do that, then the patch doesn't buy you much.
> 
> Hey, I like this! It's a subtle encouragement for developers to
> initialize all their instance variables in their __init__ or __new__
> method, with a (modest) performance improvement for a carrot. (Though
> I have to admit I have no idea how you do it. Wouldn't the set of dict
> keys be different while __init__ is in the middle of setting the
> instance variables?)
> 
> Another question: a common pattern is to use (immutable) class
> variables as default values for instance variables, and only set the
> instance variables once they need to be different. Does such a class
> benefit from your improvement?

I'm not sure who "you" is in your e-mail, but AFAICT Mark's patch
doesn't special-case __init__ or __new__. Any attribute setting on an
instance uses the shared keys array on the instance's type. "Missing"
attributes on an instance are simply NULL pointers in the instance's
values array.

(I've suggested that the keys array be bounded in size, to avoid
pathological cases where someone (ab)uses instances as fancy dicts and
puts lots of random data in them)

Regards

Antoine.