Generator comprehensions -- patch for compiler module

[Jeff Epler]

The [rejected] PEP is http://python.org/peps/pep-0289.html
It was written by Raymond Hettinger, not by me.

As a statement,
        [yield None for x in []]
is a pretty dumb one.  It will create a generator object and then
consume zero of the zero items the generator would produce.  It was an
example of the simplest generator comprehension I could think of.

Some more useful generator comprehensions might be
        cartesian_prod = [yield (x,y) for x in l1 for y in l2]
or replacing some functions from itertools:
        primes = imap(lambda x: x*x, itertools.ifilter(isprime, zstar()))
becomes
        primes = [yield x*x for x in zstar() if isprime(x)]
[zstar() being an iterator returning the natural numbers beginning at 0,
after the mathematical notation Z* for the set {0, 1, 2, ...}]  If
listcomps are better than map() and filter(), then surely a gencomp is
better than imap() and ifilter()!

The PEP doesn't really list any good use-cases, and I don't have them
either.  I simply wanted to give a response to one portion of the BDFL's
rejection:

        I expect it will take a lot of work to hack it into the code
        generator: it has to create a separate code object in order to
        be a generator.  List comprehensions are inlined, so I expect
        that the generator comprehension code generator can't share
        much with the list comprehension code generator.

My "solution" to the problem does indeed share a fair amount of code
with the existing listcomp code (and more can be shared if you don't
mind introducing 'if gencomp: else:' a little more).  I suspect it took
me as little as 4 hours (less than that, except I stared at two fairly
stupid bugs for longer than I should have).  If the time required for the
C implementation is wildly higher than that, it may be a sign that Python
should ditch the compiler-in-C as soon as there's a good way to do so.

I suppose I'd also like to disagree with Guido on another point:

        And this for something that's not that common and easily done by
        writing a 2-line helper function.  IOW the ROI isn't high enough.

If listcomps are better than map() and filter(), then surely a gencomp is
better than imap() and ifilter()!  And similarly, you don't *need* a
listcomp, because it's equivalent to a 2-line helper loop.

Jeff