Newbie tip: Prg speed from ~5 hrs to < 1 minute

[Fernando Pérez]

Terry Byrne wrote:

> One of my log files was ~60,000 lines long. At first I was using
> re.search to find the relevant error codes. But it took ~5 hours to
> process this log file -- too slow. Knowing that the string module is
> much faster than re, I switched to string.find(), and only if that
> found a certain error code would I run the re operation. (re is needed
> to obtain more specific error information.) That cut my time down to
> ~1 hr and 45 minutes, a big improvement but still too slow.
> 
> Then I used the keyword "pass" whenever I ran an re since each line
> can contain only one error; after running an re it's not necessary to
> look for other error codes at all. Now that 60,000-line log file takes
> less than a minute for Python to handle. Good enough.
> 

why don't you post some code snippet illustrating the before/after? I'm sure 
quite a few would like to see the actual details.

cheers,

f.