Performance of lists vs. list comprehensions

[Gerald Britton]

Interestingly, I scaled it up to a million list items with more or
less the same results.  It's helpful to see that your results are
different.  That leads me to suspect that mine are somehow related to
my own environment.

Still I think the key is the overhead in calling next() for each item
in the generator expression.  That in itself probably accounts for the
differences since function calls are somewhat expensive IIRC.

On Tue, Jan 19, 2010 at 11:18 AM, Stephen Hansen <apt.shansen at gmail.com> wrote:
> On Tue, Jan 19, 2010 at 7:30 AM, Gerald Britton <gerald.britton at gmail.com>
> wrote:
> [snip]
>>
>> mystring = '\n'.join( line for line in lines if <some conditions
>> depending on line> )
>
> Note, this is not a list comprehension, but a generator comprehension.
> A list comprehension is used to build, in one sweep, a list and return it.
> A generator comprehension is used to build an generator that can be iterated
> over to produce a sequence of items.
> I think you're seeing not performance of the expression, but the function
> call overhead which generators include. A generator requires one to call its
> next method to get each item: a list comprehension is just syntactical sugar
> for a for loop.
> As to which is faster, I think it depends. Your test-case is using *really*
> small ranges-- just ten items. In this case, just doing a simple loop to
> build a list and then pass it through join is probably faster. If you're
> using a much larger list though, the characteristics of the problem may
> change, where the lazy evaluation of a generator expression may be more
> desirable.
> A list comprehension includes a waste of memory, too: you have to build up a
> complete list before you return it, and if you have a lot of lines? That can
> be a problem.
> As you can see, the performance characteristics between the two narrow
> considerably if you compare a larger sample:
>  >>> Timer("' '.join([str(x) for x in l])", 'l = xrange(100)').timeit()
> 50.092024087905884
>>>> Timer("' '.join(str(x) for x in l)", 'l = xrange(100)').timeit()
> 54.591049909591675
> --S



-- 
Gerald Britton