itertools.izip brokeness

[Tom Anderson]

On Wed, 4 Jan 2006, Raymond Hettinger wrote:

> rurpy at yahoo.com wrote:
>
>> The whole point of using izip is to make the code shorter, more 
>> concise, and easier to write and understand.
>
> That should be the point of using anything in Python.  The specific goal 
> for izip() was for an iterator version of zip().  Unfortunately, neither 
> tool fits your problem.  At the root of it is the iterator protocol not 
> having an unget() method for pushing back unused elements of the data 
> stream.

An unget() isn't absolutely necessary - another way of doing it would be a 
hasNext() method, as in Java, or a peek(), which gets the next item but 
doesn't advance the iterator.

Here's some code (pardon the old-fashioned functional style in the 
iter_foo methods):

import operator

class xiterable(object):
 	"""This is an entirely abstract class, just to document the
 	xiterable interface.

 	"""
 	def __iter__(self):
 		"""As in the traditional iterable protocol, returns an
 		iterator over this object. Note that this does not have to
 		be an xiterator.

 		"""
 		raise NotImplementedError
 	def __xiter__(self):
 		"""Returns an xiterator over this object.

 		"""
 		raise NotImplementedError

class xiterator(xiterable):
 	"""This is an entirely abstract class, just to document the xiter
 	interface.

 	The xiterable methods should return self.
 	"""
 	def hasNext(self):
 		"""Returns True if calling next would return a value, or
 		False if it would raise StopIteration.

 		"""
 		raise NotImplementedError
 	def next(self):
 		"""As in the traditional iterator protocol.

 		"""
 		raise NotImplementedError
 	def peek(self):
 		"""Returns the value that would be returned by a call to
 		next, but does not advance the iterator - the same value
 		will be returned by the next call to peek or next. If a
 		call to next would raise StopIteration, this method
 		raises StopIteration.

 		"""
 		raise NotImplementedError

def xiter(iterable):
 	if (hasattr(iterable, "__xiter__")):
 		return iterable.__xiter__()
 	else:
 		return xiterwrapper(iter(iterable))

class xiterwrapper(object):
 	def __init__(self, it):
 		self.it = it
 		self.advance()
 	def hasNext(self):
 		return hasattr(self, "_next")
 	def next(self):
 		try:
 			cur = self._next
 			self.advance()
 			return cur
 		except AttributeError:
 			raise StopIteration
 	def peek(self):
 		try:
 			return self._next
 		except AttributeError:
 			raise StopIteration
 	def advance(self):
 		try:
 			self._next = self.it.next()
 		except StopIteration:
 			if (hasattr(self, "_next")):
 				del self._next
 	def __xiter__(self):
 		return self
 	def __iter__(self):
 		return self

def izip_hasnext(*xiters):
 	xiters = map(xiter, xiters)
 	while True:
 		if (reduce(operator.and_, map(hasnext, xiters))):
 			yield tuple(map(getnext, xiters))
 		else:
 			raise StopIteration

def hasnext(xit):
 	return xit.hasNext()

def getnext(it):
 	return it.next()

def izip_peek(*xiters):
 	xiters = map(xiter, xiters)
 	while True:
 		z = tuple(map(peek, xiters))
 		map(advance, xiters)
 		yield z

def peek(xit):
 	return xit.peek()

def advance(xit):
 	return xit.advance()

Anyway, you get the general idea.

tom

-- 
I am the best at what i do.