Performance of lists vs. list comprehensions

[Stephen Hansen]

On Tue, Jan 19, 2010 at 7:30 AM, Gerald Britton <gerald.britton at gmail.com>wrote:
[snip]

> mystring = '\n'.join( line for line in lines if <some conditions
> depending on line> )
>

Note, this is not a list comprehension, but a generator comprehension.

A list comprehension is used to build, in one sweep, a list and return it.

A generator comprehension is used to build an generator that can be iterated
over to produce a sequence of items.

I think you're seeing not performance of the expression, but the function
call overhead which generators include. A generator requires one to call its
next method to get each item: a list comprehension is just syntactical sugar
for a for loop.

As to which is faster, I think it depends. Your test-case is using *really*
small ranges-- just ten items. In this case, just doing a simple loop to
build a list and then pass it through join is probably faster. If you're
using a much larger list though, the characteristics of the problem may
change, where the lazy evaluation of a generator expression may be more
desirable.

A list comprehension includes a waste of memory, too: you have to build up a
complete list before you return it, and if you have a lot of lines? That can
be a problem.

As you can see, the performance characteristics between the two narrow
considerably if you compare a larger sample:

 >>> Timer("' '.join([str(x) for x in l])", 'l = xrange(100)').timeit()
50.092024087905884
>>> Timer("' '.join(str(x) for x in l)", 'l = xrange(100)').timeit()
54.591049909591675

--S
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20100119/a4ff146d/attachment-0001.html>