[Tutor] list comprehension, testing for multiple conditions

[Alan Gauld]

On 22/08/12 22:23, Pete O'Connell wrote:

>> What makes you say it is "terribly slow"? Perhaps it is as fast as it
>> could be under the circumstances. (Maybe it takes a long time because
>> you have a lot of data, not because it is slow.)
>
> OK maybe I am wrong about it being slow (I thought for loops were
> slower than lis comprehensions).

For loops are slightly slower than list comprehensions but depending on
what is happening inside the loop/comprehension the difference may not
be significant.
 > But I do know I need it to be as fast
 > as possible if I need to run it on a thousand files each with hundreds
 > of thousands of lines

If you need it as "fast as possible" you need to run it on a 
supercomputer with a program in assembler after many hours of fine 
tuning. But that will cost a lot of money and you probably don't really 
need it.

For example how often do you need to run this process? Is it a one off - 
set it running over lunch and it will likely be done by the time you get 
back. Is it an hourly event? Set it running in the background while you 
carry on processing the results of the previous run.

Steven's point is that usually absolute speed is not as important as 
people think. Modern computers are absurdly fast. When I started using 
Linux it took about 2 hours to build the kernel. Now I can build it in 
about 2 minutes! Processing a thousand files is largely limited by the 
speed of your hard drive rather than the speed of the processing on the CPU.

HTH
-- 
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/