optimization pointers?

[r.e.s.]

"Mel Wilson" <mwilson at the-wire.com> wrote ...

> One simple improvement is to get rid of
> 
>         if an_item not in a_list
> 
> which does a linear search over the list.  
> A dictionary is faster.
> 
> Without a full benchmark, it seems that 
  [...]
> will show about a x6 speedup.

I really appreciate the suggestions -- both yours 
and Anton's.  

After making the changes you indicated, the code 
runs in about 1/500th the time of the original 
(i.e. about 2 sec per MB for strings in RAM).  The 
sets idea also speeds things up tremendously, but 
not as much -- it takes about 70% longer than the 
dictionary approach.

Thanks again.
--
r.e.s.