Please comment on Draft PEP for Enhanced Generators

[Raymond Hettinger]

"Jonathan Hogg" <jonathan at onegoodidea.com> wrote in message
news:B87DF841.3E89%jonathan at onegoodidea.com...
> On 30/1/2002 8:48, in article a38c1u$945$1 at bob.news.rcn.net, "Raymond
> Hettinger" <othello at javanet.com> wrote:
>
> > I have written a draft PEP summarizing proposed enhancements to
generators:
> > http://users.javanet.com/~othello/download/genpep.htm
>
> Hi Raymond,
>
> My comments:
>
> * New lazy builtins - yup. Except, I'm not convinced of using sys.maxint
>   in index(). I know that's a big number, but I still get the worrying
>   feeling that limiting the upper bound is a bad idea. Python 2.2 will
>   automagically switch to using longs if you hit maxint, but here it will
>   just terminate without any obvious error message or idea as to why.

No one except Guido seems to like xrange(sys.maxint), so I'll change the
code for index to compute its own count.

>
> * I also prefer using 'indexed' to 'index'. It just seems to read better
>   to me. Also, I use 'index' as a variable name all the time and wouldn't
>   want the name clash.

Okay, I changed this one too.

>
> * I'm a big fan of generator comprehensions for symmetry, but I must admit
>   that I'm not sure they're worth adding new syntax for.
>
> * I'm not convinced about the suggested mechanism for implementing
consumers
>   via re-using the yield keyword with some magic. It just doesn't feel
right
>   to me. It looks too arbitrary [crisps shouldn't melt, they should
crunch].
>   I think you should present an alternative or two. Perhaps introducing
new
>   syntax for consumers, maybe:
>
>     def foo():
>         while 1:
>             x = accept
>             print x
>
>   or some such. Perhaps you should also provide a stronger argument as to
>   why consumers would be a valuable addition. Picking something from the
>   standard library and re-writing it is always good. Have you tried the
>   "nine out of ten cats understood what this code sample did" test?

I experimented with the 'accept' syntax a month ago and found that it was
full of unpleasant surprises.  In the interests of full disclosure, I've
expanded
the PEP to include what I learned.

And yes, it's hard to see why lazy consumers are useful until you need one.
I'm adding a link to Dr. Mertz's article so that a clear example is
available.

Sooner or later, you'll feel compelled to write yield None in one of your
programs.  That is precisely the cue that you are going to need the proposed
feature.  Trust me, using global variables to communicate with a generator
bites the big one.  Setting up tests to trigger an exception through a
.next()
is an even bigger mess.

I will build some sample code to show compelling uses. And I will try
your nine-out-of-ten cats test, but I'll rig the result by stating the
question
thusly:
    Which version is the easiest to understand?

    1. The one with the .submit() and .throw() syntax
    2. The one with classic generators, globals variables, and strange
        exception tests.
    3. The one built with classes that have to save a complex execution
state
        and variable state and have clumsy mechasisms for triggering mid
        stream exceptions.
    4. The one where you had to completely change your frame of reference
        and core concept so that the consumer code is written as the main
        execution body and what was the main version being re-cast as
        a generator.  Imagine writing a filelike output stream as your main
        code body and then having the inputs to stream pushed down into
        generators or iterable classes.  Yuck!
    5. The one which uses the Threading module, Mutexes, and Semaphores
        to do it the traditional way.

   Also, which would you least mind being asked to debug?


>
> * Is there a reference implementation for the consumer modification?
>

Not yet.

> Jonathan
>


Raymond Hettinger