remove duplicates from list *preserving order*

[Larry Bates]

Take a look at this recipe on ASPN:

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/204297

I think it might help.

Larry Bates


Steven Bethard wrote:
> I'm sorry, I assume this has been discussed somewhere already, but I 
> found only a few hits in Google Groups...  If you know where there's a 
> good summary, please feel free to direct me there.
> 
> 
> I have a list[1] of objects from which I need to remove duplicates.  I 
> have to maintain the list order though, so solutions like set(lst), etc. 
> will not work for me.  What are my options?  So far, I can see:
> 
> def filterdups(iterable):
>     result = []
>     for item in iterable:
>         if item not in result:
>             result.append(item)
>     return result
> 
> def filterdups(iterable):
>     result = []
>     seen = set()
>     for item in iterable:
>         if item not in seen:
>             result.append(item)
>             seen.add(item)
>     return result
> 
> def filterdups(iterable):
>     seen = set()
>     for item in iterable:
>         if item not in seen:
>             seen.add(item)
>             yield item
> 
> Does anyone have a better[2] solution?
> 
> STeve
> 
> [1] Well, actually it's an iterable of objects, but I can convert it to 
> a list if that's helpful.
> 
> [2] Yes I know, "better" is ambiguous.  If it helps any, for my 
> particular situation, speed is probably more important than memory, so 
> I'm leaning towards the second or third implementation.