[New-bugs-announce] [issue7244] itertools.zip_longest behaves strangely with an iterable class

[Daniel Urban]

New submission from Daniel Urban <urban.dani+py at gmail.com>:

I'm trying to write an iterable class, and it behaves strangely with
itertools.zip_longest. The following example demonstrates this:

class Repeater: # this class is similar to itertools.repeat
   def __init__(self, o, t):
       self.o = o
       self.t = int(t)
   def __iter__(self): # its iterator is itself
       return self
   def __next__(self):
       if self.t > 0:
           self.t -= 1
           return self.o
       else:
           raise StopIteration

(Of course this is a trivial class, which could be substituted with
itertools.repeat, but I wanted to keep it simple for this example.)

The following code shows my problem:
>>> r1 = Repeater(1, 3)
>>> r2 = Repeater(2, 4)
>>> for i, j in zip_longest(r1, r2, fillvalue=0):
...     print(i, j)
...
1 2
1 2
1 2
0Traceback (most recent call last):
 File "<stdin>", line 2, in <module>
 File "zip_longest_test_case.py", line 30, in __next__
   raise StopIteration
StopIteration
>>>

It seems, that zip_longest lets through the StopIteration exception,
which it shouldn't. 

The strange thing is, that if I use the python implementation of
zip_longest, as it is in the documentation [1], I get the expected
result:

# zip_longest as it is in the documentation:
def zip_longest(*args, fillvalue=None):
   # zip_longest('ABCD', 'xy', fillvalue='-') --> Ax By C- D-
   def sentinel(counter = ([fillvalue]*(len(args)-1)).pop):
       yield counter()         # yields the fillvalue, or raises IndexError
   fillers = repeat(fillvalue)
   iters = [chain(it, sentinel(), fillers) for it in args]
   try:
       for tup in zip(*iters):
           yield tup
   except IndexError:
       pass

Test code again:
>>> r1 = Repeater(1, 3)
>>> r2 = Repeater(2, 4)
>>> for i, j in zip_longest(r1, r2, fillvalue=0):
...     print(i, j)
...
1 2
1 2
1 2
0 2

I would think, that this is the expected behaviour.

Also, Matthew Dixon Cowles discovered, that if using list(), the C
implementation of itertools.zip_longest also works fine:

>>> r1=Repeater(1,3)
>>> r2=Repeater(2,5)
>>> list(itertools.zip_longest(r1,r2,fillvalue=0))
[(1, 2), (1, 2), (1, 2), (0, 2), (0, 2)]

This is strange, and I think it really shouldn't work this way. 
(Thanks for Matthew Dixon Cowles' help on the python-help mailing list.)

I'm attaching a test script, which tries all 4 variations (library
zip_longest with and without list(), and the documentation's zip_longest
impplementation with and without list()). 

And another thing: it works fine in 2.6.4 (with izip_longest).

----------
components: Extension Modules
files: zip_longest_test_case.py
messages: 94746
nosy: durban
severity: normal
status: open
title: itertools.zip_longest behaves strangely with an iterable class
type: behavior
versions: Python 3.1
Added file: http://bugs.python.org/file15238/zip_longest_test_case.py

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue7244>
_______________________________________