remove list elements..

[J. Clifford Dyer]

On Fri, Oct 05, 2007 at 07:27:39AM -0700, Abandoned wrote regarding remove list elements..:
> 
> Hi..
> I have a problem..
> list1=[11, 223, 334, 4223...] 1 million element
> list2=[22,223,4223,2355...] 500.000 element
> 
> I want to difference list1 to list2 but order very importent..
> 
> My result must be:
> list3=[11,334,...]
> 
> I do this use FOR easly but the speed very imported for me. I want to
> the fastest method please help me.
> 
> I'm sorry my bad english.
> 
> King regards..
> 
> -- 
> http://mail.python.org/mailman/listinfo/python-list

If you're using a recent python, you can create a set from list2, and preserve order with list1 like so:

set2 = set(list2)
list3 = [ x for x in list1 if x not in set2 ]

That will speed up your access to list2 (for 500,000 scans) at the low cost of reading through the whole thing once.

You say order is very important, but if your order is known to be strictly increasing (as it appears from your sample code), you can speed up further by doing:

set1 = set(list1)
set2 = set(list2)
list3 = sorted(set1 - set2)


Though to tell the truth, I'm not sure this actually would be faster, since you access each element of list1 just once in the list comprehension version.  And I can't be bothered to try it out with timeit.

Cheers,
Cliff