Iterator / Iteratable confusion

[Michael Spencer]

Terry Reedy wrote:
>> "Michael Spencer" <mahs at telcopartners.com> wrote in message 

> 
> We are both interested in the murky edges at and beyond conventional usage.
> 
...
> I am quite aware that multiple iterators for the same iterable (actual or 
> conceptual) can be useful (cross products, for example).  But I am dubious 
> that initialized clones of 'iterators' are *more* useful, especially for 
> Python, than multiple iterators derived from repeated calling of the 
> callable that produced the first iterator.

I'm not sure they are.  In the one 'real' example I posted on infinite series, I 
implemented the approach you advocate here.  But I'm keeping copyable iterators 
in mind.


> Here are some related reasons why I think it useful if not essential to 
> restrict the notion of iterator by restricting  iterator.__iter__ to 
> returning self unmodified.
> 
> Leaving Python aside, one can think of iterable as something that 
> represents a collection and that can produce an iterator that produces the 
> items of the collection one at a time.  In this general conceptioning, 
> iterables and iterators seem distinct (if one ignores self-iterables).

The separation is appealing, but blurrier in practice, I believe.  Neither 
itertools.cycle nor itertools.tee fits cleanly into this model.  Neither do the 
self-iterables, as you point out.

> ... giving iterators an __iter__ method, while quite useful, erases 
> (confuses) the (seeming) distinction, but giving them a minimal __iter__ 
> does so minimally, keeping iterators a distinct subcategory of iterable. 

Iterators that could not be presented to other functions for filtering or 
whatnot would be pretty limited. Unless every iterator is to be derived from 
some special-cased object, how could they not have an __iter__ method?
I accept your point that keeping the functionality of iterator.__iter__ minimal 
and predicatable limits the confusion between iterators and iterables.  But 
since that distinction is already blurred in several places, I don't find that 
argument alone decisive.

 > ...
> 
> Taking Python as it is, a useful subcategory of iterable is 'reiterable'. 
> This is distinct from iterator strictly defined.

What about itertools.cycle?  Not strictly an iterator?

   This we have iterables
> divided into iterators, reiterables, and other.  I think this is 
> didactically useful.  Spencerators are reiterables.

They may be: they are no more and no less than a thought experiment in which 
iterator.__iter__ does not return self unmodified.

> 
> Iter(iterator) returning iterator unchanged makes iterator a fixed point of 
> iter.  It ends any chain of objects returned by repeated iter calls. 
> Spencerators prolong any iter chain, making it infinite instead of finite. 
> Essential?  Repeat the paragraph above with 'a fixed point' substituted for 
> 'minimal'.
> 
I don't understand this point except in the loosest sense that deviating from
the iterator protocol makes it harder to reason about the code.  Do you mean 
something more specific?

I have been thinking about iterator.__iter__ rather like object.__new__.  Not 
returning a new instance may be surprising and inadvisable in most cases.  But 
still there are accepted uses for the technique.  Do you think these cases are 
comparable?

Do you see the iterator protocol as the vanguard of a new set of python 
protocols that are more semantically restictive than the "mapping, container, 
file-like object etc..." interfaces?  Defining iterator method semantics 
strictly seems like a departure from the existing situation.

Cheers

Michael