Iterators membership testing

[Chris Angelico]

On Sun, Aug 9, 2015 at 7:55 PM, Pierre Quentel <pierre.quentel at gmail.com> wrote:
> Thanks for the explanation. I understand that an iterator can't test membership any other way, but I'm still worried about how the documentation explains it. Reading it, I naively expected that an iterator which produces the integer 0 such as the one included in my post would say that "0 in iterator" is True, because "some value z with 0 == z is produced while iterating over y".
>
> Shouldn't it be more clear if the documentation said something like : "For user-defined classes which do not define __contains__() but do define __iter__(), x in y consumes the iterator y until some value z with x == z is produced" ?
>

Consuming the iterator is implicit in the description, because the
only way to iterate over an iterator is to call next() on it, which
consumes values. But the description covers all *iterables*, and the
caveat applies only to *iterators*. Compare:

class A:
    def __init__(self, x):
        self.x = x

    def __iter__(self):
        counter = -1
        while counter < self.x:
            counter += 1
            yield counter

This is the same code as you had, except that it's an iterable that
returns a _different_ object, and thus is not itself an iterator (in
this case, A().__iter__() returns a generator object). Note the
similarity of code to your example, and how easy it is to convert your
code to be a generator and make it reiterable. Now let's do that
membership test:

>>> iterable = A(10)
>>> for i in iterable:
...     assert i in iterable
...

No error. We can repeatedly iterate over this generator factory, and
each iteration starts a new instance of the generator, which thus
contains all the same values. Adding a print() will show that every
number is indeed iterated over.

The trap you're seeing here is that iterating over an iterator always
consumes it, but mentally, you're expecting this to be iterating over
a new instance of the same sequence. That's perfectly understandable,
but due to the extreme flexibility of the iterator and iterable
protocols, there's no way for the interpreter to say anything
different. Make your object reiterable, and then it'll behave the way
your brain is expecting... but the docs aren't incorrect here.

ChrisA