Please comment on Draft PEP for Enhanced Generators

[Raymond Hettinger]

"Michael Chermside" <mcherm at destiny.com> wrote in message
news:mailman.1012408548.6645.python-list at python.org...
> One minor detail:
>
> >  def ints(cnt=0, limit=None):
> > 'Generate an integer sequence starting a zero or cnt upto limit or
infinity'
> >         while not limit or cnt < limit:
> >     yield cnt
> >     cnt += 1
>
> Calling ints(0,0) should probably not yield the sequence:
>      0, 1, 2, 3, ...
> I would instead expect an empty sequence! I would suggest this tiny
> modification:
>
>     def ints(cnt=0, limit=None):
>        """Generate an integer sequence starting a zero or cnt upto limit
>        or infinity"""
>        while limit is None or cnt < limit:
>           yield cnt
>           cnt += 1
>

I agree; however, the BDFL just sent a note saying that ints() is
unnecessary since the same result can be achieved with xrange(sys.maxint).
So I've eliminated ints() and reposted the PEP.

>
> As for my overall thoughts:
>
>     * The new built-ins are all good ideas. Do them immediately. The only
>       unnecessary bit (that other still-under-consideration PEPs contain
>       other ways of achieving ints()) is already mentioned in your PEP.

Having these functions available is surprisingly liberating.

>     * Generator comprehensions seem to be potentially useful, though I'm
>       not really sure. The syntax is nice, the only issue is whether
>       their utility is sufficient to meet the bar to be worth extending
>       the language. I'm on the fence.

Since there is not a way to simulate generator comprehensions, it's
hard to tell whether these would become an essential part of programming
in Python or just one of those nice-to-haves sitting unused in the back
drawer.

I scanned all of my source code where list comprehensions were used
and found that in about a third of the cases, the 'yield' format would
have been preferrable.  I'm not sure how many times I used regular
generators or other approaches simply because the generator
comprehension tool didn't exist.

Normally, I feel conservative about extending the language syntax,
but this one is especially attractive since:
1 it adds functionality and simplifies coding
2 it uses existing keywords and syntax
3 it backward compatible
4 the whole mechanism is already there just waiting to be turned on
5 there is a near zero learning curve

>
>     * Two-way generator parameter passing seems awefully bold. To my
>       mind, one of the great things about generators is that they meet
>       the (very simple) definition of an iterator. With this, they no
>       longer do. I like lazy consumers -- really I do -- but I'd rather
>       be conservative about putting something like this in the language.
>

It seems bold, given how simply generators already work, but
other implementations of generators (like the thread based
generator.py) already have provisions for two-way parameter passing
so that consumers are put on an equal footing with producers.  I think
two-way is the norm, not the exception.

Besides, 98% of the mechanism is already in place.  When I first needed
to pass in information, the only available alternative was clumsy.  I had
to set a global variable, call .next(), and assign the local from the
global.

There was no work around for triggering an exception inside a generator.
This is a true deficiency.  It is the only case in Python where active code
cannot be excepted to or through.

Yield is not just a simple iterator creator.  It does something else truly
wonderful -- it suspends execution and saves state.  I think it is good
for a lot more than its original purpose.

>
> And hey... great work in writing these up!

Thanks.  I spend a month teasing these ideas out to find a proposal
that had maximum value with minimum changes to the language.
Then, I had to switch hats and honestly write-up the weak points
and well as the strong points of the design.

> -- Michael Chermside
>
>

-- Raymond Hettinger