[Python-ideas] Generators are iterators

[Ron Adam]

On 12/13/2014 01:37 PM, Andrew Barnert wrote:
>> >I don't know what thing you are referring to. If I write this:
>> >
>> >py> def gen():
>> >...     yield 1
>> >...
>> >py> it = gen()
>> >
>> >then `gen` is a function.
> Chris Barker just asked exactly the same question yesterday, except that he trimmed the quote more and used "g" instead of "it" for his example. So I'll just paste the same reply, and you can manually s/g/it/.
>
> Of course g is an instance of generator (and therefore also an iterator).
>
> But you're not defining the generator type here--that already exists as a builtin type. What you're defining is a thing made up of a code object and a frame, which is used by that generator instance to do its generating. That thing is not a generator, or a generator type (or an iterator instance or type), or a __next__ method, or anything else with a name.
>
> (I don't know if that's the thing Guido was trying to describe as not an iterator, but I don't think that's too important, since he's already acknowledged that the distinction he was trying to make isn't important anyway.)

 >>> def gen():
...     yield 1
...
 >>> type(gen)
<class 'function'>
 >>> gen
<function gen at 0x7f75425a0b70>

It's a function with a flag set.

 >>> gen.__code__.co_flags
99

 >>> dis(gen)
   2           0 LOAD_CONST               1 (1)
               3 YIELD_VALUE
               4 POP_TOP
               5 LOAD_CONST               0 (None)
               8 RETURN_VALUE

Where a function's flags would be this...

 >>> def foo():
...     return 1
...
 >>> foo.__code__.co_flags
67

So, it the answer is yes and no.  It's a function object, with an altered 
behaviour.  It seems like the flag may be checked at call time, and then a 
new function generator instance is returned..

 >>> gi = gen()
 >>> type(gi)
<class 'generator'>
 >>> gi
<generator object gen at 0x7f75409bf798>

And the original function body is moved to it's gi_code attribute.

 >>> dis(gi.gi_code)
   2           0 LOAD_CONST               1 (1)
               3 YIELD_VALUE
               4 POP_TOP
               5 LOAD_CONST               0 (None)
               8 RETURN_VALUE

I think the send() method, a builtin C routine, handles all the particulars 
of running gi_code with gi_frame.  (I'd have to look to be sure.)

I think if function definitons with yield in them created 
generator_instance_factory objects instead of function objects, things 
would be easier to explain.  What it comes down to, is they aren't normal 
functions and it's a bit confusing to think of them that way.

Cheers,
    Ron