something Evil happens when large hashes destroyed

Tim Peters tim.one at home.com
Sun Nov 18 18:39:52 EST 2001


[guuge]
> I was trying to sort about 100,000 items by splitting them into
> groups (using a python dictionary type) and then successively splitting
> these groups until they were small enough to use a brute force method.

Try list.sort().

> My test program, which used only two or three keys to create the first
> split, worked fine.  When I tried the real thing, using 256 keys, the
> program slowed to a crawl.  The python interpreter took forever to
> destroy the dictionaries.
>
> Here's a little program ...

I tried that w/ 2.2b2 under Win98SE, 866MHz, 256MB RAM, also with 100,000
elements, and the number of bins didn't matter:  for assorted values from 1
through 100,000, creation time was 2 seconds and destruction time 0
(time.time() only has 1-second resolution on Windows).

So best guess is that you're fighting your platform C library's
malloc()/free().  Which OS?  Which C compiler and library?  If you build
Python yourself, try rebuilding 2.2b2 --with-pymalloc, and see whether that
helps.  It would be interesting to know one way or the other.





More information about the Python-list mailing list