PEP 289: Generator Expressions (please comment)

Mon Oct 27 11:49:56 EST 2003

On Monday 27 October 2003 04:52 pm, Holger Krekel wrote:
> Alex Martelli wrote:
> > > Although most people on python-dev and here seems to like this PEP I
> > > think it generally decreases readability.  The above constructs seem
> > > heavy and difficult to parse and thus i am afraid that they interfere
> > > with the perceived simplicity of Python's syntax.
> >
> > Let's see: today, I could code:
> >     sum([x*x for x in roots])
> > In 2.4, I will be able to code instead:
> >     sum(x*x for x in roots)
>
> Yes, Raymond send me some private mail already and i clarified
> that i don't like list comprehension syntax as well :-)
>
> Actually i do like generator expressions *more* than list comprehensions
> so i agree to your points that gen expressions do improve on the (rather
> specific) list comprehensions.

And therefore, since you ain't gonna take list comprehensions away,
you should be in favour, not against, generator expressions.

Personally, I agree with David Ascher's evaluation in the intro to Chapter 1
of "Python Cookbook" (end of p.2): "List comprehensions have clearly
become wildly successful".  I have seen non-programmers just learning
programming and Python together "GET" list comprehensions AT ONCE,
particularly if they had ever seen typical set notation as used in maths,
but not only under such conditions.  I do agree with you that generator
expressions are a further improvement (except in name -- sigh -- I wish
"iterator expressions" had prevailed instead!-).

> Nevertheless, i have the impression that this kind of syntax (be it list
> comprehension or generator expressions) somewhat decreases the
> chances for "immediate" understanding of python code without knowing
> the language much.  It's likely that you need to parse generator

I've had substantial feedback from articles on Python I had written,
particularly on Italian magazines, and much of it was from people who
indeed don't know the language much (or at all).  That feedback tells
me that list comprehensions were absolutely nothing like the obstacle
you fear, confirming my experiences with newbies.  The things that DID
give non-Python-knowers _serious_ trouble were rather e.g.:

    while <condition>:
       <body>
    else:
       <something else>

NOBODY who doesn't have a good grasp of Python gets "immediate
understanding" of this -- *not one person* out of all I've talked about
this.  A large majority imagines that the "else:" clause is executed if
the <condition> NEVER holds, the remaining minority just doesn't
get it at all, NOBODY guesses that this may have to do with whether
a "break" inside the <body> has or hasn't executed.

> expressions in detail especially because they don't provide sharp
> visual clues like indented blocks do.  I do consider the "visual clues"
> to be a strength of current python although you can always provide
> examples with a language's syntax which are not readable.

Oh, the above example is quite readable and rich with visual clues --
it's just that its _semantics_ are totally anti-intuitive to all readers who 
haven't studied Python pretty well.  On the contrary, and I wish there
was a way to make a bet on this and settle it objectively, I claim e.g.:

    print sum(x*x for x in thelist if x>10)

will be "immediately read" as "print the sum of the squares of those
elements of 'thelist' that are larger than 10" by a majority of the
readers (if anything, understandable confusion will come about the
fact that "print" has nothing whatsoever to do with the *printer*
that one may or may not have attached to one's computer, but
rather will normally display stuff on a terminal-window... the mental
disconnection of "print" from "printer" is NOT intuitive at all -- it just
SEEMS that way after one has been programming long enough!).

Of course, more complicated expressions will be harder to read --
but that's quite independent of whether those expressions will
be generator expressions or other kinds.

> Generator expressions will probably raise the barrier to understanding
> python code without having appropriate knowledge but that may very well
> prove to be worth it. We'll probably find out soon so i think we don't
> need to argue a lot about it now.  IOW, i am not completly opposed
> to the PEP just a bit skeptical.  

We'll find out in, I suspect, well over a year (when 2.4 final is released
and gets widespread).  And we'll "find out" on a purely anecdotal and
experiential basis, just as we have "found out" about list comprehensions
over the last few years, unless somebody has money burning in their
pocket to finance a systematic study, which I seriously doubt:-).

> And, btw,  i appreciated very much,
> that Guido made it clear from the start that Python 2.3 would be a
> maintenance release with no changes to syntax.  I am probably just

So did I!  Great move.

> astonished at these huge syntax-threads all over python-dev :-)

I was anything but astonished -- a bunch of language enthusiasts
stopped from any _productive_ discussion of syntax changes for,
what, over two years (since the last alpha of 2.2 until now)...?!  It's
not amazing that we'd be bubbling over now we have a chance...:-).

Guido (with support from the usual adverse-to-any-change brigade)
will no doubt exert his usual veto right with the usual firmness, so
all of our threads are little more than a venting out of too-long
repressed energy, anyway:-).

Alex