PEP 255: Simple Generators

[Glyph Lefkowitz]

On Sun, 24 Jun 2001, Tim Peters wrote:

> [Glyph Lefkowitz]
> > If python's answer to "what should we do with new feature X"? is
> > "add a keyword" it will turn into Perl before too long.
> 
> Since that's very rarely Python's answer, this is slippery-slope FUD shouted
> from a flat plane.

My point is that 'yield' is a keyword, and that new keywords are bad.  
Although there has been a Pronouncement on this :) I still disagree with
it; it would not require a keyword addition if creating a generator were
something like

  import generator

  def _genfunc():
    for i in range(100):
     generator.yield(i)

  gen = generator.create(_genfunc)

But again, given that there's been a pronouncement, this is just
rearranging the deck chairs on the titanic, so I'll be quiet now :).

> Generators are much simpler and often suffice.

Point taken.  Upon reflecting, 'for' does not preclude the inclusion of
'while' loops; for myself, I really can't see generators as anything but
an aberrant special case of coroutines, but the fact that the language
treats them differently doesn't necessarily mean they'll be broken.  And
it does seem that many others see them as independantly useful.

> > (or should we have syntax for that too?  'task' anyone?)
> 
> That's something for the microthreads PEP to stake out.  You know that
> implementing the semantics efficiently in Jython too has been a severe
> impediment to Stackless adoption, right?

I know that, but I don't agree with it.  Jython can't do even-based I/O,
but the 'select' module remains in the Python standard library.  The JVM
is kinda crappy; if I wanted to be bound to the limitations of Java,
there's another language I could program in, thank you very much :)

> Generator semantics are simple and synchronous.
> ...
> For the same reasons, Python would retain "while" loops even if Guido
> added a "goto".

> The audience for microthreads is enthusiastic but small, and the best
> shot it has is to ride in as a generalization of something else.

I had been thinking the opposite way... shame on me for doubting the
wisdom of guido :)

> If you're eager to ignore the *pragmatics* of generators, then I'm
> happy to repay the favor wrt microthreads.

I'll have to give generators the benefit of the doubt.

> Neil's implementation is so elegant it makes me smile -- calling it a
> hack is at best in bad taste.

> If you looked into it instead of ranting about it, you'd discover that
> the things he did to the eval loop are necessary preconditions for
> implementing microthreads too -- the ability to suspend a frame in
> midstream is fundamental to all these gimmicks.  This will be the end
> of the path for most, but it takes you closer to your goal.

Although I'd read it before, reading it again with this comment in mind
(while chanting "refactor, refactor, refactor" to myself in a trancelike
state) makes it much clearer to me how such a transformation would be
made.  And that it would, in fact, be easier to make it from Neil's
patches than without them...

> Generators aren't becoming central to Python; they're an especially
> powerful instance of 2.2's iterator interface (a point you should
> consider: generators live in the iterator world, not in the thread
> world, and iterators are ubiquitous in all kinds of apps; thread-like
> control flow has a much smaller audience, and smaller still if the
> "threads" block at synchronous I/O calls, and smaller still again if
> they don't act like "real threads" when running in foreign code).

Thanks for illustrating this.  FWIW, I retract my objections to PEP 255;
your description of why it's useful has been very helpful and educational.

but-I-still-can't-entirely-forgive-guido-for-print>>-and-lambda-ly y'rs,

                      ______      __   __  _____  _     _
                     |  ____ |      \_/   |_____] |_____|
                     |_____| |_____  |    |       |     |
                     @ t w i s t e d m a t r i x  . c o m
                     http://twistedmatrix.com/users/glyph