Iterator class to allow self-restarting generator expressions?

[Gabriel Genellina]

En Sun, 01 Mar 2009 13:20:28 -0200, John O'Hagan <research at johnohagan.com>  
escribió:

> Inspired by some recent threads here about using classes to extend the
> behaviour of iterators, I'm trying to replace some some top-level  
> functions
> aimed at doing such things with a class.
>
> So far it's got a test for emptiness, a non-consuming peek-ahead method,  
> and
> an extended next() which can return slices as well as the normal mode,  
> but
> one thing I'm having a little trouble with is getting generator  
> expressions
> to restart when exhausted. This code works for generator functions:

[...]

> I'd like to do the same for generator expressions, something like:
>
> genexp = (i for i in range(3))
>
> regenexp = Regen(genexp, restart=True)
>
> such that regenexp would behave like reg, i.e. restart when exhausted  
> (and
> would only raise StopIteration if it's actually empty). However because
> generator expressions aren't callable, the above approach won't work.

I'm afraid you can't do that. There is no way of "cloning" a generator:

py> g = (i for i in [1,2,3])
py> type(g)()
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: cannot create 'generator' instances
py> g.gi_code = code
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: readonly attribute
py> import copy
py> copy.copy(g)
Traceback (most recent call last):
...
TypeError: object.__new__(generator) is not safe, use generator.__new__()
py> type(g).__new__
<built-in method __new__ of type object at 0x1E1CA560>

You can do that with a generator function because it acts as a "generator  
factory", building a new generator when called. Even using the Python C  
API, to create a generator one needs a frame object -- and there is no way  
to create a frame object "on the fly" that I know of :(

py> import ctypes
py> PyGen_New = ctypes.pythonapi.PyGen_New
py> PyGen_New.argtypes = [ctypes.py_object]
py> PyGen_New.restype = ctypes.py_object
py> g = (i for i in [1,2,3])
py> g2 = PyGen_New(g.gi_frame)
py> g2.gi_code is g.gi_code
True
py> g2.gi_frame is g.gi_frame
True
py> g.next()
1
py> g2.next()
2

g and g2 share the same execution frame, so they're not independent. There  
is no easy way to create a new frame in Python:

py> type(g.gi_frame)()
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: cannot create 'frame' instances

One could try using PyFrame_New -- but that's way too magic for my taste...

-- 
Gabriel Genellina