something Evil happens when large hashes destroyed

Hans Nowak wurmy at earthlink.net
Sun Nov 18 12:37:03 EST 2001


guuge wrote:
> 
> I was trying to sort about 100,000 items by splitting them into
> groups (using a python dictionary type) and then successively splitting
> these groups until they were small enough to use a brute force method.
> 
> My test program, which used only two or three keys to create the first
> split, worked fine.  When I tried the real thing, using 256 keys, the
> program slowed to a crawl.  The python interpreter took forever to
> destroy the dictionaries.
> 
> Here's a little program that will place X number of elements into
> Y number of bins and gives the time used to do the placement and
> the time used to destroy the dictionary.
> 
[snip snip]

Interesting... on my box, a P3/900 with 128 Mb, the dicts are destroyed
faster when I use more keys. Low numbers of keys result in long
destruction times.

For 100,000 elements, I always get a creation time of 2 seconds, and
the following destruction times:
20 keys: 61 seconds
34 keys: 24 seconds
60 keys: 8 seconds
256 keys: 3 seconds

Of course, this could have been timed better, by using a larger set than
100,000. Maybe some other time...

It seems, the larger the dicts, the more time it takes to destroy it.
In other words, if N elements are distributed over a lot of small
dict, the total destroy time is a lot lower than when they are
distributed over a few large dicts. Maybe this makes sense for people
who know the C implementation of the dict... (I don't. :-)

Anyway, my guess is that if you use a lot of keys, the total destroy
time should be lower... correct?



More information about the Python-list mailing list