Iterator class to allow self-restarting generator expressions?

[John O'Hagan]

On Sun, 1 Mar 2009, Terry Reedy wrote:
> John O'Hagan wrote:
> > Inspired by some recent threads here about using classes to extend the
> > behaviour of iterators, I'm trying to replace some some top-level
> > functions aimed at doing such things with a class.
> >
> > So far it's got a test for emptiness, a non-consuming peek-ahead method,
> > and an extended next() which can return slices as well as the normal
> > mode, but one thing I'm having a little trouble with is getting generator
> > expressions to restart when exhausted. This code works for generator
> > functions:
> >
> > class Regen(object):
> >     """Optionally restart generator functions"""
> >     def __init__(self, generator, options=None, restart=False):
> >         self.gen = generator
>
> Your 'generator' parameter is actually a generator function -- a
> function that created a generator when called.
>
> >         self.options = options
>
> Common practice would use 'args' instead of 'options'.
>
> >         self.gen_call = generator(options)
>
> If the callable takes multiple args, you want '*options' (or *args)
> instead of 'options'.
>
> That aside, your 'gen_call' parameter is actually a generator -- a
> special type of iterator (uncallable object with __next__ (3.0) method).
>
> It is worthwhile keeping the nomenclature straight.  As you discovered,
> generator expressions create generators, not generator functions.  Other
> than being given the default .__name__ attribute '<genexpr>', there is
> otherwise nothing special about their result.  So I would not try to
> treat them specially.  Initializing a Regen instance with *any*
> generator (or other iterator) will fail.
>
> On the other hand, your Regen instances could be initialized with *any*
> callable that produces iterators, including iterator classes.  So you
> might as well call the parameters iter_func and iterator.
>
> In general, for all iterators and not just generators, reiteration
> requires a new iterator, either by duplicating the original or by saving
> the values in a list and iterating through that.
>

Thanks to all who replied for helping to clear up my various confusions on 
this subject. For now I'm content to formulate my iterators as generator 
function calls and I'll study the various approaches offered here. For now 
here's my attempt at a class that does what I want:

class Exgen(object):
    """works for generator functions"""
    def __init__(self, iter_func, restart=False, *args):
        self.iter_func = iter_func
        self.args = args
        self.iterator = iter_func(*args)
        self.restart = restart
        self._buffer = []
        self._buff()
        
    def __iter__(self):
        return (self)
    
    def __nonzero__(self):
        if self._buffer:
            return True
        return False

    def _buff(self, stop=1):
        """Store items in a list as required"""
        for _ in range(stop - len(self._buffer)):
            try:
                self._buffer.append(self.iterator.next())
            except StopIteration:
                if self.restart:
                    self.iterator = self.iter_func(self.args)
                    self._buffer.append(self.iterator.next()) 
                else:
                    break

    def peek(self, start=0, stop=1):
        """See a slice of whats coming up"""
        self._buff(stop)
        return self._buffer[start:stop]

    def next(self, start=0, stop=1):
        """Consume a slice"""
        self._buff(stop)
        if self._buffer:
            result = self._buffer[start:stop]
            self._buffer = self._buffer[:start] + self._buffer[stop:]
            return result
        else:
            raise StopIteration
            
    def clear(self):
        """Empty the buffer"""
        self._buffer = []
        self._buff()

Regards,

John