Best way to handle large lists?

[Chaz Ginger]

Paul Rubin wrote:
> Sybren Stuvel <sybrenUSE at YOURthirdtower.com.imagination> writes:
>>> I don't know enough about Python internals, but the suggested
>>> solutions all seem to involve scanning bigList. Can this presumably
>>> linear operation be avoided by using dict or similar to find all
>>> occurrences of smallist items in biglist and then deleting those
>>> occurrences?
>> And how would that beat O(n)? Every element of bigList has to be
>> scanned at one point, either to compare it to every earlier element in
>> bigList and eliminate it, or to compare it to every element in
>> smallList.
> 
> Maybe the application should use sets instead of lists for these
> collections.
What would sets do for me over lists?

Chaz