[Python-ideas] Introduce collections.Reiterable

[Steven D'Aprano]

On Fri, Sep 20, 2013 at 11:03:17AM +0100, Paul Moore wrote:

> An iterable is an object that returns an iterator when passed to
> iter(). It's *iterators* that have to have __next__, not iterables. An
> iterable hast to have __iter__, which as far as I know dict views do

It is not correct to say that iterables have to have an __iter__ method, 
by both common usage of the term, and by the definition in the glossary. 
Sorry to repeat myself, but iterables can also be objects which obey the 
sequence protocol. I already showed an example of a non-pathological 
object which can be iterated over but where isinstance(obj, Iterable) 
returns the wrong result. See my previous post, or the end of this one, 
for that example.

Here's the glossary entry in full:


iterable
    An object capable of returning its members one at a time. Examples 
of iterables include all sequence types (such as list, str, and tuple) 
and some non-sequence types like dict, file objects, and objects of any 
classes you define with an __iter__() or __getitem__() method. Iterables 
can be used in a for loop and in many other places where a sequence is 
needed (zip(), map(), ...). When an iterable object is passed as an 
argument to the built-in function iter(), it returns an iterator for the 
object. This iterator is good for one pass over the set of values. When 
using iterables, it is usually not necessary to call iter() or deal with 
iterator objects yourself. The for statement does that automatically for 
you, creating a temporary unnamed variable to hold the iterator for the 
duration of the loop. See also iterator, sequence, and generator.

http://docs.python.org/3.4/glossary.html

I think we can all agree that dict views are iterables. You can iterate 
over them, and they have an __iter__ method. We can also agree that 
views aren't sequences, nor are they iterators themselves:

py> keys = {}.keys()
py> iter(keys) is keys
False

and isinstance gives the correct result for views:

py> isinstance(keys, Sequence)
False
py> isinstance(keys, Iterator)
False
py> isinstance(keys, Iterable)
True

None of this is in dispute! But (and this was really a very minor point 
in my original post, seemingly blown all out of proportion) you can't 
iterate over a view directly. Or perhaps, for the avoidance of doubt, I 
should say you can't iterate over a view *manually* without creating an 
itermediate iterator object. Iteration in Python is implemented by two 
protocols:

1) the iterator protocol, which repeatedly calls __next__ until 
StopIteration is raised; and

2) the sequence protocol, which repeatedly calls __getitem__(0), 
__getitem__(1), __getitem__(2), ... until IndexError is raised.

Dict views don't obey either of these, as it has no __next__ or 
__getitem__ method. That is all I mean when I say that dict views aren't 
"directly [manually] iterable". Instead, they have an __iter__ method 
which returns an object which is directly iterable, a dict_keyiterator 
object.

This really was a very minor point, I've already spent far more words 
on this than it deserves. But the important point seems to have been 
missed, namely that the Iterable ABC gives the wrong result for some 
objects which are iterable. Here it is again:

py> class Seq:
...     def __getitem__(self, index):
...             if 0 <= index < 5: return index+1000
...             raise IndexError
...
py> s = Seq()
py> isinstance(s, Iterable)  # The ABC claims Seq is not iterable.
False
py> for x in s:  # But it actually is.
...     print(x)
...
1000
1001
1002
1003
1004


Can anyone convince me this is not a bug in the Iterable ABC?



-- 
Steven