[Tutor] removing consecutive duplicates from list

[Peter Otten]

On 20/04/2021 18:48, Manprit Singh wrote:

> Consider a list given below:
> lst = [2, 3, 3, 4, 5, 5, 3, 7, 9, 9, 4]
> i need to remove consecutive duplicates from list lst:
> the answer must be :
> 
> [2, 3, 4, 5, 3, 7, 9, 4]
> 
> The code that i have written to solve it, is written below:
> lst = [2, 3, 3, 4, 5, 5, 3, 7, 9, 9, 4]
> ls = lst[1:]+[object()]
> [x for x, y in zip(lst, ls) if x != y]
> 
> The list comprehension gives the desired result. just need to know if this
> 
> program can be done in a more readable and less complex way.

The itertools module has many tools that you can use to deal with 
problems like the above:

The obvious one:

 >>> lst = [2, 3, 3, 4, 5, 5, 3, 7, 9, 9, 4]
 >>> wanted = [2, 3, 4, 5, 3, 7, 9, 4]

 >>> from itertools import zip_longest
 >>> wanted == [x for x, y in zip_longest(lst, lst[1:]) if x != y]
True

You may want to use islice instead of an actual slice.


A variant that will continue to work if you replace the list with an 
iterator (e. g. a file):

 >>> from itertools import tee
 >>> a, b = tee(lst)
 >>> next(b)
2
 >>> [x for x, y in zip_longest(a, b) if x != y]
[2, 3, 4, 5, 3, 7, 9, 4]
 >>> wanted == _
True

A conceptually different one, treating consecutive dupes as groups:

 >>> from itertools import groupby
 >>> wanted == [k for k, g in groupby(lst)]
True

Pick your favourite ;)