Getting values out of a CSV

[Gabriel Genellina]

En Fri, 13 Jul 2007 09:05:29 -0300, Daniel <no at no.no> escribió:

>>> > Note that every time you see [x for x in ...] with no condition, you
>>> can
>>> > write list(...) instead - more clear, and faster.
>>> >
>>> > data = list(csv.reader(open('some.csv', 'rb')))
>>>
>>> Faster? No. List Comprehensions are faster.
>>
>> On my system just putting into a list is faster.  I think this is
>> because you don't need to assign each line to the variable 'line' each
>> time in the former case.
>
> I don't know why there seems to be a differece, but I know that list  
> comps
> are python are very heavily optimised.

In principle both ways have to create and populate a list, and a list  
comprehension surely is better than a loop using append() - but it still  
has to create and bind the intermediate variable on each iteration.
I think that testing with a csv file can't show the difference between  
both ways of creating the list because of the high overhead due to csv  
processing.
Using another example, with no I/O involved (a generator for the first  
10000 fibonacci numbers):

C:\TEMP>python -m timeit -s "import fibo" "list(fibo.fibo())"
10 loops, best of 3: 39.4 msec per loop

C:\TEMP>python -m timeit -s "import fibo" "[x for x in fibo.fibo()]"
10 loops, best of 3: 40.7 msec per loop

(Generating less values shows larger differences - anyway they're not  
terrific)

So, as always, one should measure in each specific case if optimization is  
worth the pain - and if csv files are involved I'd say the critical points  
are elsewhere, not on how one creates the list of rows.

-- 
Gabriel Genellina