Best way to handle large lists?

[Paul Rubin]

Chaz Ginger <cginboston at hotmail.com> writes:
> I have a system that has a few lists that are very large (thousands or
> tens of thousands of entries) and some that are rather small. Many times
> I have to produce the difference between a large list and a small one,
> without destroying the integrity of either list. I was wondering if
> anyone has any recommendations on how to do this and keep performance
> high? Is there a better way than
> 
> [ i for i in bigList if i not in smallList ]

diff = list(set(bigList) - set(smallList))

Note that doesn't get you the elements in any particular order.