PEP 288: Generator Attributes

[Terry Reedy]

"John Roth" <johnroth at ameritech.net> wrote in message
news:uv94kkq5ajbae3 at news.supernews.com...
>
> "Rocco Moretti" <roccomoretti at netscape.net> wrote
> > process of set attributes -> call processing function -> repeat is
an
> > akward way of achieving what you really want to do - continue the
> > generator with given data. I'd agree with what Guido said on
> > Python-Dev: there is no advantage in this scheme over passing a
> > mutable dummy object when the generator is created (you could even
> > call it __self__ if you really wanted too ...).

> I thoroughly agree.

Me too.

> with some bemusement, because, for me at least, the most
> intuitive thing to do is make yield a built in function rather than
> a statement.

But then we would need a another keyword (possible, of course) to tell
the compiler to change the generated bytecode.  It would also break
the parallel between 'return something' and 'yield something'.  The
only difference in execution is that yield does less -- by *not*
deleting the execution frame.

> That way, it can return a value, which is what
> was passed in on the reinvocation.

One of the design features of generators is that resumption is
extremely quick precisely because the argument processing and function
setup steps are bypassed.  Just restore the execution frame and go on
with the next statement [bytecode actually].

Another design feature is that they are meant to work seemlessly with
for statements, with only one explicit call (to produce the iterator
that 'for' needs).  In this context, passing in additional values is
not possible.

Another problem is with multiple yields.  Consider the following
hypothetical generator with the proposed yield() 'function' (see next
comment for why the '' marks):

def genf(a):
  b=yield(a)
  b,c=yield(a+b)
  yield(a+b+c)
gen = genf(1)

Then the documentation must say: on the first call to gen.next(), pass
one arg; on the next, pass two; finally, don't pass any.  Not good,
methings.

> The trouble with the whole thing is that conceptually, "yield" has
two
> values: one that it returns to the caller, and one that it
> gets from the caller. It's a two way pipe.

(You mean, 'would be'.)  Pipes and such usually have explicit read and
write methods that make the order of operations clear.  The yield()
proposal hijacks function notation to squash a read and write
together -- with the order switched at the two ends.

To explain: y=f(x) usually means 'set y to a value depending on
(calculated from) x'  -- this is the meaning of 'function'.
Operationally, a function call means to send x to process f to
initialize the corresponding parameter,  suspend operation while f
operates, and set y to the value returned by f.  The combined
next(arg)/yield() idea warps and shuffles these semantics.

Let p = pipe or whatever:  Then (omitting suspend) y = yield(x) could
mean
  p.write(x); y=p.read()
where (contrary to the implication of function notation) the value
read cannot depend on x since it is calculated at the other end before
the write.

The corresponding x = gen.next(y) then means
  x=p.read; p.write(y)
which makes the order of value passing the opposite of what a function
call means.  The only reason to write y is for a possible future call
to gen.next().  I think it much better to separate the read and write
and pair the writing of y with the reading of the x that functionally
depends on the value written.

One could reverse the meaning of x = gen.next(y) to be
  p.write(y); x=p.read()
but the x read would still not depend on the y written since the
latter would have to be set aside and ignored until the predetermined
x was sent back.  IE, y=yield(x) would have to mean and be implemented
as
  <hidden-slot> = p.read(); p.write(x); y=<hidden-slot>

In either case, there is a synchronization problem in that the values
sent to the generator are shifted by one call.  The last gen.next()
call must send a dummy that will be ignored.  On the other hand, if,
for instance, there is one yield function which depends on the
variable reset by the yield function, then that variable must be
separately initialized in the initial genf() call.

So it arguably makes just as much sense to initialize via genf() with
a mutable with paired write/read, send/receive, or set/get methods or
the equivalent operations (as with dicts).  One can then make explicit
calls to pass data in either direction without fake 'function' calls.
If one pairs 'yield None' with 'dummy=gen.next()' or 'for dummy in
genf(mutable):', one can even do all value passing, in both
directions, via the two-way channel or mutable and use 'yield'
strictly for suspend/resume synchronization of the coroutines.

--
This proposal come close to asking for what the original Stackless did
with continuations.  These allowed some mind-boggling code.  Almost
too fancy for what Python is meant to be 8-).

Terry J. Reedy