python disk i/o speed

[Peter Hansen]

nnes wrote:
> 
> Peter Hansen <peter at engcorp.com> wrote in message news:<3D525996.AA4F55EF at engcorp.com>...
> to suggestions.
> >
> > Why don't you run the profiler on it and see where the time is
> > being spent?  Random arbitrary optimizations is a waste of developer
> > time, and generally makes the code less maintainable.
> 
> well, I am not as proficient using the profiler as you may asume :-).
> Maybe you can explain what the output of it means. As far as I can see
> it tells me what I already know: there is a script called bench3.py
> with a function filterfile2() which does some i/o using strings.

I'm not sure you can tell more than this without breaking the function
into several functions, actually.  Sorry about that.  It seems the
profiler as it works right now only gives granularity down to the
call level, not to individual lines.

On the other hand, the results look strange.  Are you accidentally
running the code twice somehow?  Maybe that's part of the problem.
I don't think you should get two lines here:

> Fri Aug 09 09:27:05 2002    filtprof
> Fri Aug 09 09:27:05 2002    filtprof

And this looks like everything is included twice:

>          6 function calls in 65.583 CPU seconds
> 
>    Ordered by: cumulative time
> 
>    ncalls  tottime  percall  cumtime  percall filename:lineno(fu
>         2    0.005    0.003   65.583   32.792 profile:0(filterfi
>         2    0.000    0.000   65.578   32.789 <string>:1(?)
>         2   65.578   32.789   65.578   32.789 bench3.py:3(filter
>         0    0.000             0.000          profile:0(profiler

Are you doing more than just "profile.run('filterfile2(some args...)')" ?

Your interpretation is slightly off though.  The <string> part
refers to the highest level, where the command string is interpreted 
in the run() call (I think).  Then profile calls filterfile.... 
ah, I think I see where the double call comes from.  You are
calling filterfile() at the end of the module without protecting
it with "if __name__ == '__main__':" per convention.  I think
you are getting one run accidentally, then explicitly calling
filterfile2() a second time.  Maybe you're doing this also 
in the regular file, not just in the profiling?  (Prob'ly not...)

-Peter