unique elements from list of lists

Tekkaman simone.leo at gmail.com
Fri Feb 9 05:33:52 EST 2007


I have a list of lists and I want to define an iterator (let's call
that uniter) over all unique elements, in any order. For example,
calling:

sorted(uniter([['a', 'b', 'd'], ['b', 'c'], ['a', 'c', 'd']]))

must return ['a', 'b', 'c', 'd']. I tried the following
implementations:

from itertools import chain
def uniter1(listOfLists):
    for item in set(chain(*listOfLists)): yield item

def uniter2(listOfLists):
    for item in reduce(
        lambda x,y: x|y,
        [set(list_) for list_ in listOfLists]
        ): yield item

speed test with timeit says the first one is slightly faster. What
bothers me is that it builds a set from an iterator and then another
iterator from the set. Is there a way to implement this using only
iterators? I also tried a "full python" implementation (by that I mean
one that does not use the built-in set and keeps track of previously
yielded items in a list) but the one I pulled out is about 180 times
slower. Here it is:

def uniter3(listOfLists):
    done = []
    for list_ in listOfLists:
        for item in list_:
            if not item in done:
                done.append(item)
                yield item

Thanks in advance for any contribution.

-- Simone




More information about the Python-list mailing list