generators (was RE: [Python-Dev] Re: Sets: elt in dict, lst.include)

[Tim Peters]

[Neil Schemenauer]
> What's the chances of getting generators into 2.2?

Unknown.  IMO it has more to do with generalizing the iteration protocol
than with generators per se (a generator object that doesn't play nice with
"for" is unpleasant to use; otoh, a generator object that can't be used
divorced from "for" is frustrating too (like when comparing the fringes of
two trees efficiently, which requires interleaving two distinct traversals,
each naturally recursive on its own)).

> The implementation should not be hard.  Didn't Steven Majewski have
> something years ago?

Yes, but Guido also sketched out a nearly complete implementation within the
last year or so.

> Why do we always get sidetracked on trying to figure out how to
> do coroutines and continuations?

Sorry, I've been failing to find a good answer to that question for a decade
<0.4 wink>.  I should note, though, that Guido's current notion of
"generator" is stronger than Icon/CLU/Sather's (which are "strictly
stack-like"), and requires machinery more elaborate than StevenM (or Guido)
sketched before.

> Generators would add real power to the language and are simple
> enough that most users could benefit from them.  Also, it should be
> possible to design an interface that does not preclude the
> addition of coroutines or continuations later.

Agreed.

> I'm not volunteering to champion the cause just yet.  I just want
> to know if there is some issue I'm missing.

microthreads have an enthusiastic and possibly growing audience.  That gets
into (C) stacklessness, though, as do coroutines.  I'm afraid that once you
go beyond "simple" (Icon) generators, a whole world of other stuff gets
pulled in.  The key trick to implementing simple generators in current
Python is simply to decline decrementing the frame's refcount upon a
"suspend" (of course the full details are more involved than *just* that,
but they mostly follow *from* just that).

everything-is-the-enemy-of-something-ly y'rs  - tim