optimization pointers?

[Anthony McDonald (at in)]

"Mel Wilson" <mwilson at the-wire.com> wrote in message
news:xIf2/ks/Kvyd089yn at the-wire.com...
>
>    Nice touch!  I tried slices and took a huge performance
> hit (almost 3x the list version) but I didn't use `for ... in
> xrange ...`.  It must have all been in the while-loop test
> and index incrementing.
>
>         Regards.        Mel.

Thanks.

My original attempt which incidentally used range was about half a second
slower than yours with 700K of test data. Just as I was about to close the
editor window I noticed your return statement.

return len (voc.keys())

Which creates a list of keys to then apply len to, but thats an unneeded
step as len applied directly to a dictionary returns the number of keys. I
made the change and gained 7 hundreds of a second. Not much, still behind
yours, but it suggested that chainging range to xrange and avoiding the list
creation might help. Viola!

The interesting thing that benchmarking with test data shows, is that the
difference in speed between our 2 routines is about 0.05secs per 700K
processed. That difference remains constant at least upto 80Mb of input
data, implying that slices are no more efficent than string appending, and
may actually in this case be less efficent.

Anthony McDonald