itertools.izip brokeness

[Raymond Hettinger]

Paul Rubin wrote:
> What do you think of my suggestion of passing an optional arg to the
> StopIteration constructor, that the caller can use to resume the
> iterator or take other suitable recovery steps?  Maybe this could
> interact with PEP 343 in some useful way.

Probably unworkable.  Complex to explain and use.   Makes the API
heavy.  Hard to assure retro-fitting for every possible kind of
iterator or iterable.

Am not sure of the best solution:

1. Could add an optional arg to zip()/izip() with a mutable container
to hold a final, incomplete tuple:  final=[]; zip(a,b,leftover=final).
This approach is kludgy and unlikely to lead to beautiful code, but it
does at least make accessible data that would otherwise be tossed.

2. Could add a new function with None fill-in -- essentially an
iterator version of map(None, a,b).  Instead of None, a user specified
default value would be helpful in cases where the input data stream
could potentially have None as a valid data element.  The function
would also need periodic signal checks to make it possible to break-out
if one the inputs is infinite. How or whether such a function would be
used can likely be answered by mining real-world code for cases where
map's None fill-in feature was used.

3. Could point people to the roundrobin() recipe in the
collections.deque docs -- it solves a closely related problem but is
not exactly what the OP needed (his use case required knowing which
iterator gave birth to each datum).

4. Could punt and leave this for straight-forward while-loop coding.
Though the use case seems like it would be common, there may be a
reason this hasn't come up since zip() was introduced way back in
Py2.0.

5. Could create an iterator wrapper that remembers its last accessed
item and whether StopIteration has been raised.  While less direct than
a customized zip method, the wrapper may be useful in contexts other
than zipping -- essentially, anywhere it is inconvenient to have just
consumed an iterator element.  Testing the wrapper object for
StopIteration would be akin to else-clauses in a for-loop. OTOH, this
approach is at odds with the notion of side-effect free functional
programming and the purported benefits of that programming style.



Raymond Hettinger