[Python-Dev] cpython: Implement PEP 412: Key-sharing dictionaries (closes #13903)

[Kristján Valur Jónsson]

Probably any benchmark involving a large amount of object instances with non-trivial dictionaries.

Benchmarks should measure memory usage too, of course.  Sadly that is not possible in standard
cPython.  Our 2.7 branch has extensive patching to allow custom memory allocators to be used
(it even eliminates the explicit "malloc" calls used here and there in the code) and exposes some
functions, such as sys.getpymalloced(), useful for memory benchmarking.

Perhaps I should write about this on my blog.  Updating the memory allocation macro layer in
cPython for embedding is something I'd be inclined to contribute, but it will involve a large amount
of bikeshedding, I'm sure :)

Btw, this is of great interest to me at the moment, our Shanghai engineers are screaming at the
memory waste incurred by dictionaries.  A 10 item dictionary consumes 1/2k on 32 bits, did you 
know this?

K

> -----Original Message-----
> From: python-dev-bounces+kristjan=ccpgames.com at python.org
> [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On
> Behalf Of R. David Murray
> Sent: 23. apríl 2012 21:56
> To: Antoine Pitrou
> Cc: python-dev at python.org
> Subject: Re: [Python-Dev] cpython: Implement PEP 412: Key-sharing
> dictionaries (closes #13903)
> 
> On Mon, 23 Apr 2012 22:22:18 +0200, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Mon, 23 Apr 2012 17:24:57 +0200
> > benjamin.peterson <python-checkins at python.org> wrote:
> > > http://hg.python.org/cpython/rev/6e5855854a2e
> > > changeset:   76485:6e5855854a2e
> > > user:        Benjamin Peterson <benjamin at python.org>
> > > date:        Mon Apr 23 11:24:50 2012 -0400
> > > summary:
> > >   Implement PEP 412: Key-sharing dictionaries (closes #13903)
> >
> > I hope someone can measure the results of this change on real-world
> > code. Benchmark results with http://hg.python.org/benchmarks/ are not
> > overly promising.
> 
> I'm pretty sure that anything heavily using sqlalchemy will benefit, so that
> would be a good place to look for a real-world benchmark.
> 
> --David
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-
> dev/kristjan%40ccpgames.com