can list comprehensions replace map?

[Peter Otten]

Andrew Dalke wrote:

> Steven Bethard wrote:
>> Here's one possible solution:
>> 
>> py> import itertools as it
>> py> def zipfill(*lists):
>> ...  max_len = max(len(lst) for lst in lists)
> 
> A limitation to this is the need to iterate over the
> lists twice, which might not be possible if one of them
> is a file iterator.
> 
> Here's a clever, though not (in my opinion) elegant solution
> 
> import itertools
> 
> def zipfill(*seqs):
>     count = [len(seqs)]
>     def _forever(seq):
>         for item in seq: yield item
>         count[0] -= 1
>         while 1: yield None
>     seqs = [_forever(seq) for seq in seqs]
>     while 1:
>         x = [seq.next() for seq in seqs]
>         if count == [0]:
>             break
>         yield x
 

> This seems a bit more elegant, though the "replace" dictionary is
> still a bit of a hack
> 
> from itertools import repeat, chain, izip
> 
> sentinel = object()
> end_of_stream = repeat(sentinel)
> 
> def zipfill(*seqs):
>     replace = {sentinel: None}.get
>     seqs = [chain(seq, end_of_stream) for seq in seqs]
>     for term in izip(*seqs):
>         for element in term:
>             if element is not sentinel:
>                 break
>         else:
>             # All sentinels
>             break
>             
>         yield [replace(element, element) for element in term]

Combining your "clever" and your "elegant" approach to something fast
(though I'm not entirely confident it's correct):

def fillzip(*seqs):
    def done_iter(done=[len(seqs)]):
        done[0] -= 1
        if not done[0]:
            return
        while 1:
            yield None
    seqs = [chain(seq, done_iter()) for seq in seqs]
    return izip(*seqs)

Whether we ran out of active sequences is only tested once per sequence.

Fiddling with itertools is always fun, but feels a bit like reinventing the
wheel in this case. The only excuse being that you might need a lazy
map(None, ...) someday...

Peter