delete duplicates in list

[Martin Chilvers]

Maybe I'm just old-school, and it won't win any prizes for speed but...

>>> a = [1, 3, 2, 3, 4, 2, 1, 4, 3, 2, 1, 2, 4]
>>> reduce(lambda l, x: x not in l and l.append(x) or l, a, [])
[1, 3, 2, 4]

;^)

Martin

> -----Original Message-----
> From: python-list-bounces+martin=enthought.com at python.org 
> [mailto:python-list-bounces+martin=enthought.com at python.org] 
> On Behalf Of Bengt Richter
> Sent: Thursday, October 30, 2003 5:26 PM
> To: python-list at python.org
> Subject: Re: delete duplicates in list
> 
> 
> On Wed, 29 Oct 2003 22:02:08 +0100, christof hoeke 
> <csad7 at yahoo.com> wrote:
> 
> >hello,
> >this must have come up before, so i am already sorry for asking but a
> >quick googling did not give me any answer.
> >
> >i have a list from which i want a simpler list without the 
> duplicates 
> >an easy but somehow contrived solution would be
> >
> > >>> a = [1, 2, 2, 3]
> > >>> d = {}.fromkeys(a)
> > >>> b = d.keys()
> > >>> print b
> >[1, 2, 3]
> >
> >there should be an easier or more intuitive solution, maybe 
> with a list
> >comprehension=
> >
> >somthing like
> >
> > >>> b = [x for x in a if x not in b]
> > >>> print b
> >[]
> >
> >does not work though.
> >
> If you want to replace the original list without a temporary 
> new list, and your original is sorted (or you don't mind 
> having it sorted), then you could do the following (not 
> tested beyond what you see ;-), which as an extra benefit 
> doesn't require hashability:
> 
>  >>> def elimdups(thelist):
>  ...     thelist.sort()   # remove if you just want to 
> eliminate adjacent duplicates
>  ...     i = 0
>  ...     for item in thelist:
>  ...         if item==thelist[i]: continue
>  ...         i += 1
>  ...         thelist[i] = item
>  ...     del thelist[i+1:]
>  ...
>  >>> a = [1, 2, 2, 3]
>  >>> elimdups(a)
>  >>> a
>  [1, 2, 3]
>  >>> a=[]
>  >>> elimdups(a)
>  >>> a
>  []
>  >>> a = [123]
>  >>> elimdups(a)
>  >>> a
>  [123]
>  >>> a = ['a', ['b', 2], ['c',3], ['b',2], 'd']
>  >>> a
>  ['a', ['b', 2], ['c', 3], ['b', 2], 'd']
>  >>> elimdups(a)
>  >>> a
>  [['b', 2], ['c', 3], 'a', 'd']
>  >>> a = list('deaacbb')
>  >>> elimdups(a)
>  >>> a
>  ['a', 'b', 'c', 'd', 'e']
> 
> Not sure how this was decided, but that's the way it works:
>  >>> 'a' > ['b', 2]
>  True
> 
> Hm, it would have been nicer to have an optional sort flag as
> a second parameter. Oh, well, another time...
> 
> Regards,
> Bengt Richter
> -- 
> http://mail.python.org/mailman/listinfo/python-list
>