[Python-ideas] Add lookahead iterator (peeker) to itertools

[Masklinn]

On 2013-02-26, at 11:37 , Serhiy Storchaka wrote:

> On 26.02.13 12:06, Masklinn wrote:
>> Although I can trivially get a segfault with islice:
>> 
>>     from itertools import islice, count
>> 
>>     it = count()
>>     while True:
>>         it = islice(it, 0)
>> 
>> Run this, C-c at some point, every CPython version on my machine
>> segfaults (pypy does not)
> 
> Thank you. This looks as pretty same issue as with tee(). I knew that there must be more such catches, but could not find them.
> 
> http://bugs.python.org/issue17300
> 

Cool.

And I looked back in the VCS, turns out the code hasn't been lost but
the issue was not in the stdlib, it was a custom iterator (used as a
wrapper for a bunch of operations) which needed to be reapplied very
often (code basically went Iterator -> mix of tee, chain and
dropwhile -> Iterator -> same mix), essentially doing the following:

import itertools

_placeholder = object()
class It(object):
    def __init__(self, stream):
        self.stream = iter(stream)
        self.stopped = False
    def __iter__(self):
        return self
    def __next__(self):
        if self.stopped: raise StopIteration()
        val = next(self.stream, _placeholder)
        if val is _placeholder:
            self.stopped = True
            raise StopIteration()
        return val

it = itertools.count()
while True:
    it = It(it)
    next(it)

I'm not sure if there's any way to implement such a wrapping iterator in
a way which does not ultimately blow the stack (save in C taking after
itertools implementations I guess as they don't seem to have the issue)