map del efficiency python2.2 and 2.1

[Tim Peters]

[Andrew McNamara]
> ...
> I wrote a very simple benchmark that allocated and deallocated a million
> small objects on a 600MHz P3 running linux 2.4.18 and glibc-2.2.4 with
> enough memory to prevent paging.
>
> Results and code are below. It's clear that the system malloc hurts
> badly for deallocation, but even with pymalloc deallocation is still
> an expensive exercise (the attribute dictionary, presumably). Slots are
> very efficient for small objects, in terms of CPU and memory footprint.

pymalloc in (all flavors of) 2.2 was still an experimental feature (that's
why it's disabled by default), and wasn't tuned for the way new-style
classes work.  In particular, pymalloc's "small object threshold" is much
too low in the 2.2 line, so low that *no* instance dict can get handled by
it, and consequently it still passes on lots of work to the system
malloc/free.  The strong benefit you got from it is more indirect than you
may think; your platform free() appears to be truly dreadful when dealing
with lots of small objects.

Under current CVS Python (what will eventually be Python 2.3), pymalloc is
tuned properly for new-style classes, and the deletion times are minor on my
home Win98SE 866MHz box (the del times below are 100x smaller than you were
seeing with 221+pymalloc); they should become minor on your box too, as
pymalloc's code is the same on all platforms, while its few hardcoded
assumptions about OS paging are geared more toward Unix than Windows:

C:\Code\python\PCbuild>python -O temp.py
testing slots
 create + setattr took 13.1s
 setattr took 1.9s
 del took 0.6s
testing obj
 create + setattr took 36.6s
 setattr took 2.1s
 del took 1.1s
testing trad
 create + setattr took 30.8s
 setattr took 2.4s
 del took 1.0s

This was leaving gc enabled, but changing your testit() to

def testit(ctor, count):
    print "testing", ctor.__name__

    start()
    a = map(ctor, xrange(count))
    stop(' create + setattr')
    gc.collect()

    start()
    aa = ctor(0)
    map(aa.set, xrange(count))
    stop(' setattr')
    del aa
    gc.collect()

    start()
    del a
    gc.collect()
    stop(' del')

because it just confuses things to time explicit gc.collect() calls too
(except when you're specifically trying to time collection, as you may have
intended to do in your " del" test).  gc is a bit zippier in current CVS
too.