[Python-ideas] Introduce collections.Reiterable

Fri Sep 20 05:18:54 CEST 2013

You're right, but first I'm not sure that I agree with Terry's comment that
"if dev needs to iterate an input more than once, the specification should
say so. If the user wants to pass an iterator, the user can instead pass
list(iter). The reason to have user rather than dev make this call is that
user is in a better position than dev to know whether iter is effectively
finite."

The problem with this is that it's exhausting to keep checking whether a
function needs an iterable or not, and it's noisy for the user of the
function to have to cast to things list.  I've had silent breakages when I
change the return value of one function from list to generator not
realizing that it was passed somewhere to another function that wanted a
reiterable.  Most importantly, there's no sure *and* easy way to assert
that the input a function is reiterable, and so the silent breakages are
hard to discover.  Even if the user should be the one deciding what to do,
the dev has to be able to assert that the right thing was done.

Therefore, I feel there should be a definitive test for reiterability.
 Either:
* The documentation should promise that Iterable and not Iterator is
reiterable, or
* Reiterable should be added to collections.abc, or
* some other definitive test that hasn't been brought up yet.

Best,

Neil

On Thu, Sep 19, 2013 at 10:19 PM, Andrew Barnert <abarnert at yahoo.com> wrote:

> On Sep 19, 2013, at 15:22, Neil Girdhar <mistersheik at gmail.com> wrote:
>
> Why not do it the way Antoine suggested, but instead of
>
> self.need_cloning = isinstance(it, collections.Iterator)
>
> have
>
> self.need_cloning = isinstance(it, collections.Reiterable)
>
>
> Because we already have Iterator today, and we don't have Reiterable, and
> nobody has yet come up with a useful case where the latter would do the
> right thing and the former wouldn't.
>
> (Also because the second one is the exact opposite of what you meant...
> But I assume that's a simple typo.)
>
>
> Then, mark the appropriate classes as subclasses of collections.Reiterable
> where collections.Sequence < collections.Reiterable < collections.Iterable?
>
> Best,
>
> Neil
>
>
> On Thu, Sep 19, 2013 at 5:40 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>> On 9/19/2013 8:18 AM, Steven D'Aprano wrote:
>>
>>> On Thu, Sep 19, 2013 at 07:12:26PM +1000, Nick Coghlan wrote:
>>>
>>>  is there any obvious case where "iterable but
>>>> not an iterator" gives the wrong answer?
>>>>
>>>
>>> I'm not sure if it counts as "obvious", but one can write an iterator
>>> that is re-iterable. A trivial example:
>>>
>>> class Reiter:
>>>      def __init__(self):
>>>          self.i = 0
>>>      def __next__(self):
>>>          i = self.i
>>>          if i < 10:
>>>              self.i += 1
>>>              return i
>>>          self.i = 0
>>>
>>
>> This, I agree, is bad.
>>
>>
>>           raise StopIteration
>>>      def __iter__(self):
>>>          return self
>>>
>>>
>>> I know that according to the iterator protocol, such a re-iterator
>>> counts as "broken":
>>>
>>> [quote]
>>> The intention of the protocol is that once an iterator’s next() method
>>> raises StopIteration, it will continue to do so on subsequent calls.
>>>
>>
>> I would add 'unless and until iter() or another reset method is called.
>> Once one pokes at a iterator with another mutation method, all bets are
>> off. I would consider Reiter less broken or not at all if the reset in
>> __next__ were removed, since then it would continue to raise until
>> explicity reset with __iter__
>>
>>
>>  Implementations that do not obey this property are deemed broken. (This
>>> constraint was added in Python 2.3; in Python 2.2, various iterators are
>>> broken according to this rule.)
>>>
>>> http://docs.python.org/2/**library/stdtypes.html#**iterator-types<http://docs.python.org/2/library/stdtypes.html#iterator-types>
>>>
>>> but clearly there is a use-case for re-iterable "things", such as dict
>>> views, which can be re-iterated over. We just don't call them iterators.
>>> So maybe there should be a way to distinguish between "oops this
>>> iterator is broken" and "yes, this object can be iterated over
>>> repeatedly, it's all good".
>>>
>>> At the moment, dict views aren't directly iterable (you can't call
>>> next() on them). But in principle they could have been designed as
>>> re-iterable iterators.
>>>
>>> Another example might be iterators with a reset or restart method, or
>>> similar. E.g. file objects and seek(0). File objects are officially
>>> "broken" iterators, since you can seek back to the beginning of the
>>> file. I don't think that's a bad thing.
>>>
>>> But nor am I sure that it requires a special Reiterable class so we can
>>> test for it.
>>>
>>>
>>>
>>
>> --
>> Terry Jan Reedy
>>
>>
>>
>> ______________________________**_________________
>> Python-ideas mailing list
>> Python-ideas at python.org
>> https://mail.python.org/**mailman/listinfo/python-ideas<https://mail.python.org/mailman/listinfo/python-ideas>
>>
>> --
>>
>> --- You received this message because you are subscribed to a topic in
>> the Google Groups "python-ideas" group.
>> To unsubscribe from this topic, visit https://groups.google.com/d/**
>> topic/python-ideas/**OumiLGDwRWA/unsubscribe<https://groups.google.com/d/topic/python-ideas/OumiLGDwRWA/unsubscribe>
>> .
>> To unsubscribe from this group and all its topics, send an email to
>> python-ideas+unsubscribe@**googlegroups.com<python-ideas%2Bunsubscribe at googlegroups.com>
>> .
>> For more options, visit https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out>
>> .
>>
>
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20130919/a8ec14b4/attachment.html>