Callable generators (PEP 288: Generator Attributes, again)

[Bengt Richter]

On 19 Nov 2003 10:28:27 -0800, michele.simionato at poste.it (Michele Simionato) wrote:

>"Francis Avila" <francisgavila at yahoo.com> wrote in message news:<vrmpthqfk8bc8b at corp.supernews.com>...
>> I'm suggesting the PEP's functionality, not its syntax and semantics.  My
>> contention is that the PEP regards generators as too class-like, when they
>> are more naturally considered as function-like.
>> 
>> For example, your iterator class/instance would look like this:
>> 
>> def iterator(x=1)(x):
>>     yield x
>>     yield x
>> 
>> print iterator.next() # -> 1
>> print iterator(5)  # -> 5
>> 
>> The local name "x" is updated (according to the second parameter list in the
>> function definition) right after the yield of the previous call when
>> iterator is called, behaving like a state-persistent callable function.  If
>> it's just "nexted", it behaves like a plain old iterator.
>
>I see what you mean, now. Still, the notation is confusing, since I
>do think an iterator as something which it obtained by "instantiating"
>a generator. On the other hand, your iterator(5) would not be a newly
>"instantiated" iterator, but the same iterator with a different
>parameter x. So, you would need a different syntax, as for instance
>iterator[5]. Still, I do think this would be confusing. The class
>solution would be more verbose but clearer.

<rant>
I really like generators, but I am not really happy with the magic transmogrification
of an ordinary function into a generator-instance factory as a magic side effect
of defining a state machine with yields in the function body. Sorry to complain
about a past decision, but one effect was also to exclude yield-ing in a subroutine
of the generator (which would in general require suspending a stack of frames, but would
open a lot of possibilities, ISTM).
</rant>

But, trying to live with, and extend, current realities, we can't call the generator
factory function name again to pass values into the generator, since that merely creates
another generator. But if we have

    gen = factory(some_arg)

where factory could be e.g., (I'm just using 'factory' instead of 'iterator' for semantics)

    def factory(x=1): yield x; yield x

we could potentially call something other than gen.next(), i.e., if gen had a gen.__call__ defined,

    gen(123)

could be a way to pass data into the generator. But what would it mean w.r.t. backwards compatibility
and what would be different? Obviously the calls to gen.next() generated by

   for x in factory(123): print x

would have to work as now. But consider, e.g.,

   for x in itertools.imap(factory(), 'abc'): print x

presumably it would be equivalent to

   gen = factory()
   for x in itertools.imap(gen, 'abc'): print x

and what we'd like is to have some local variables (just x in this case) set to whatever
is passed through gen.__call__(...) arg list, in this case successively 'a', 'b', and 'c'.

Since gen is the product of a factory (or implicit metaclass operating during instantiation
of the function??) it should be possible to create a particular call signature for gen.__call__.

One syntactic possibility would be as Francis suggested, i.e.,

    def iterator(x=1)(x): 
        yield x
        yield x

but, BTW, I guess Francis didn't mean

    print iterator.next() # -> 1
    print iterator(5)  # -> 5

since it is not iterator that has the .next method, but rather the returned
generator. So unless I misunderstand it would have to be

    gen = iterator()
    print gen.next() # -> 1
    print gen(5)  # -> 5

IIRC, this is what I was getting at originally (among other things;-)

If we live with the current parameter list definition mechanism, we are forced
to pass default or dummy values in the initial factory call, even if we intend
to override them with the very first call to gen('something'). We have the option
of using them through next(), but we don't have to. I.e.,

    gen = iterator()
    print gen('hoo') # -> hoo (default 1 overridden)
    print gen('haw') # -> haw

also, incidentally,

    gen = iterator()
    print gen('hoo') # -> hoo (default 1 overridden)
    print gen.next() # -> hoo (x binding undisturbed)

That doesn't seem so hard to understand, if you ask me. Of course,
getting gen.__call__ to rebind the local values requires real changes,
and there are some other things that might make usage better or worse
depending -- e.g., how to handle default parameters. On the call to the
factory, defaults would certainly act like now, but on subsequent gen() calls,
it might make sense to rebind only the explicitly passed parameters, so the default
would effectively be the existing state, and you could make short-arg-list calls. Hmmm...

This might be a good combination use of defaults in the factory call to set initial
defaults that then are optionally overridden in the gen() calls. I kind of like that ;-)

> 
>> Here's an efficient reversable generator:
>> 
>> def count (n=0)(step=1):
>>     while True:
>>         yield n
>>         n += step

My way would be

   def count (step=1, count=0):
       while True:
           yield n
           n += step

   ctr = count()
   ctr()   # -> 0
   ctr()   # -> 1
   ctr()   # -> 2
   ctr(-1) # -> 1
   ctr()   # -> 0
   ctr()   # -> -1

>
>Yeah, it is lightweight, but a class derived from "Iterator" 
>would be only one line longer, so I am not convinced. Also,
>people would start thinking that you can define regular
>functions with two sets of arguments, and this would generate
>a mess ...

Well, what do you think of the above?
BTW, thanks Francis for the citation ;-)

PS, I wonder if a co_yield would be interesting/useful? (Think PDP-11 jsr pc,@(sp)+ ;-)

Regards,
Bengt Richter