Populating a dictionary, fast

[Hrvoje Niksic]

Michael Bacarella <mbac at gpshopper.com> writes:

> $ uname -a
> Linux xxx 2.6.9-22.ELsmp #1 SMP Sat Oct 8 21:32:36 BST 2005 x86_64 x86_64 x86_64 GNU/Linux
>
> We've also tried it on this version (on a different machine):
>
> $ uname -a
> Linux yyy 2.6.18-8.el5 #1 SMP Thu Mar 15 19:46:53 EDT 2007 x86_64 x86_64 x86_64 GNU/Linux

Note that both machines are x86_64.  Please try your test on a 32-bit
machine and report the results.  I suspect performance won't degrade
there.

Most of the people who tried your test couldn't repeat your result
(including me), but some did.  It is quite possible that those who did
were running on variants of x86_64, just like you.  My guess is you've
hit a bug that degrades performance of hashing of longs, or of large
dicts, only affecting 64-bit architectures.

As a workaround, you can try skipping the long() construction and
simply hashing strings.  (There should be no loss of information, and
string objects are probably just as compact as Python longs in that
range.)