Generator expressions v/s list comprehensions

[Alex Martelli]

Mahesh Padmanabhan <mahesh at privacy.net> wrote:

> Hi,
> 
> When list comprehension was added to the language, I had a lot of 
> trouble understanding it but now that I am familiar with it, I am not
> sure how I programmed in Python without it. 

Oh good.

> 
> Now I see that generator expressions have been added to the language 
> with 2.4 and I question the need for it. I know that it allows for lazy
> evaluation which speeds things up for larger lists but why was it 
> necessary to add it instead of improving list comprehension? 
> 
> Was there some sort of limitation that prevented list comprehension from
> taking over the functionality that generator expressions provide?

Sure, and that limitation is: list comprehensions return lists.  This
one "limitation" (together with Python's applicative order evaluation,
and you couldn't change THAT without breaking the whole caboodle of
existing programs!) implies everything else.

> 
> I have always liked the fact that Python has limited capabilities for
> having more than one way to do it and it seem to me that generator 
> expressions break that philosophy. It is similar to how you have to use
> range or xrange depending on how large the range is.
> 
> Can someone please explain the reasoning behind it?

Generator comprehensions are wonderful and there is no way Python list
comprehensions can provide the same features, since lists need to be
lists.  Sure, list(e(x) for x in foo) IS just the same thing as [e(x)
for x in foo].  We'll remove the redundancy in 3.0 -- not earlier
because it will break backwards compatibility.  The only sensible way I
can see right now for 3.0 to remove this redundancy is by removing list
comprehensions and leaving only generator comprehensions, btw.


Alex