delete duplicates in list

[Bengt Richter]

On Wed, 29 Oct 2003 22:02:08 +0100, christof hoeke <csad7 at yahoo.com> wrote:

>hello,
>this must have come up before, so i am already sorry for asking but a 
>quick googling did not give me any answer.
>
>i have a list from which i want a simpler list without the duplicates
>an easy but somehow contrived solution would be
>
> >>> a = [1, 2, 2, 3]
> >>> d = {}.fromkeys(a)
> >>> b = d.keys()
> >>> print b
>[1, 2, 3]
>
>there should be an easier or more intuitive solution, maybe with a list 
>comprehension=
>
>somthing like
>
> >>> b = [x for x in a if x not in b]
> >>> print b
>[]
>
>does not work though.
>
If you want to replace the original list without a temporary new list,
and your original is sorted (or you don't mind having it sorted), then
you could do the following (not tested beyond what you see ;-), which
as an extra benefit doesn't require hashability:

 >>> def elimdups(thelist):
 ...     thelist.sort()   # remove if you just want to eliminate adjacent duplicates
 ...     i = 0
 ...     for item in thelist:
 ...         if item==thelist[i]: continue
 ...         i += 1
 ...         thelist[i] = item
 ...     del thelist[i+1:]
 ...
 >>> a = [1, 2, 2, 3]
 >>> elimdups(a)
 >>> a
 [1, 2, 3]
 >>> a=[]
 >>> elimdups(a)
 >>> a
 []
 >>> a = [123]
 >>> elimdups(a)
 >>> a
 [123]
 >>> a = ['a', ['b', 2], ['c',3], ['b',2], 'd']
 >>> a
 ['a', ['b', 2], ['c', 3], ['b', 2], 'd']
 >>> elimdups(a)
 >>> a
 [['b', 2], ['c', 3], 'a', 'd']
 >>> a = list('deaacbb')
 >>> elimdups(a)
 >>> a
 ['a', 'b', 'c', 'd', 'e']

Not sure how this was decided, but that's the way it works:
 >>> 'a' > ['b', 2]
 True

Hm, it would have been nicer to have an optional sort flag as
a second parameter. Oh, well, another time...

Regards,
Bengt Richter