[Python-Dev] Re: PEP 279

[Tim Peters]

[Tim]
>> Bingo.  If you want coroutines, design a coroutine facility.
>> Piling more semantics on to "yield" takes a delightfully simple yet
>> powerful gimmick and hides it under ten kinds of obscurity.

[Raymond Hettinger]
> I don't see the obscurity.

I know <wink>.  As I said earlier in the msg, "generator exceptions do less
damage (IMO) to simplicity than some of the other extensions".  I had in
mind above *all* the other gimmicks suggested to be layered on top of yield,
including burying yield in what otherwise looked like listcomps, and the
"generator parameter passing" idea.

> This is the kind of tool that is invisible and out of the way until the
> one day you really need it and the tool is right there in your hand (as
> if by time machine).
>
> Throw wouldn't be used 98% of the time.

I'd call it 99+%, but that was my original point:  it's rarely needed.  The
(very) few times I've needed it, I was happy rolling my own.

> ...
> I didn't come up with these ideas out of the ether.  It came up in a
> real world application for producing index prints from jpegs.

The question is then whether simple generators are a fork that hasn't yet
grown enough tines, or whether your app really wanted a knife.  That is, if
you try to use a feature and find it's not enough for a specific problem, it
doesn't follow that the feature is flawed.  BTW, the cleanest way I can
think of to implement generator exceptions would be to add a new opcode to
the Python virtual machine, and that's a pretty heavy change for a
rarely-needed minor convenience.

> That being said, I accept that generator parameter passing and exception
> passing are doomed.

Heh.  I started pushing for generators in the very early 90's.  The idea was
rejected multiple times, but usually because somebody else hijacked the idea
in favor of continuations instead.  Pare away the over-elaborate, and
something simple that remains may have a chance, if you just keep poking at
it for a decade <wink>.

> Here is my alternate proposal that is much simpler and doesn't encourage
> weirdness.
> [Developers feel free to stomp on the idea but please don't smack
> me around for thinking it up].
>
> Specification for Generator Attributes:
>
>     1. Allow attributes to be assigned to generator objects.

That's doable.  You can already assign attributes to generator-functions, so
I figure this means generator-iterators (to which you cannot currently
assign attributes).

>     2.  Provide a means of accessing those attributes from within the
>     generator by using a reference to __self__.
<
>  def mygen():
>      print __self__.greeting
>      yield 'hello'
>      print __self__.greeting
>
>  g = mygen()
>  g.greeting = 'Good morning'
>  print g.next()
>  g.greeting = 'Good night'

This isn't an, umm, compelling example.  Why would I *really* want to do
this?  If I really wanted to, I expect I'd again make the generator a method
of a class and use vanilla instance variables -- or maybe I would really
want coroutines.

> Advantages:  Uses the standard python attribute assignment idiom
> so there is no learning curve and no surprises.

Ditto for instance variables.

> Easily implementable without new keywords, new builtins, or parser
> magic.

That isn't so:  the parser is going to have to recognize "__self__" as a
special token, and generate special code to make it resolve to the
generator-iterator object derived from the generator-function.  This isn't
at all like the current use of "self" in methods, because in that context
"self" is just a named parameter like any other, and lives in the function's
local namespace like any other named parameter.  This new gimmick would make
more sense if the generator's 'def' were spelled

    def mygen(__self__):

but then it gets uglier if mygen is also a method.

> Provides a data sharing solution that avoids the yield / next() matching
> problem, avoids enclosing classes, and avoid global variables.  Simple
> and neat.

I don't think it's so simple, as there's nothing else like it in Python now:
it can't really be explained in terms of anything that already exists.
Names always live in the local, global or builtin namespaces.  __self__
would have to live in the local namespace to make any sense, but *no* name
shows up "by magic" in a local namespace now.

> Disads:  Introduces a new system variable, __self__.

The rub is that it's a system variable unlike any other.  You may find it's
more convenient to pass an explicit "message carrier" to mygen:

    class Carrier:
        pass  # just a dummy to hold names

    def mygen(__self__):
        __self__.this and __self__.that, exactly as above

    msgs = Carrier()
    g = mygen(msgs)
    msgs.greeting = 'Good morning'
    print g.next()

etc.  If this is actually a common pattern for you, a very simple wrapper
class could capture it, so that you could write

    g = Wrapper(mygen)
    g.greeting = 'Good morning')
    etc.


> P.S.  No one needs to say anything mean to make me go away.

Only Guido wants you to go away <wink>, and he's never mean, just Dutch.

I don't know that generators will never be enhanced, but they're a brand-new
feature to the vast bulk of Python programmers, and it would be prudent to
wait a few years to see which issues come up repeatedly and don't find a
reasonably simple solution via combining them with other Python features
(like classes indeed:  "doesn't need a class" isn't really a selling point).
It took 'em a decade to get in, and they're already more powerful than in
the languages Python borrowed them from (primarily Icon and CLU, with a dash
of Sather).