[FEEDBACK] Is this script efficient...is there a better way?

[David LeBlanc]

<snip>
> Have you actually tested this? I used to believe that larger buffers are
> always better for performance.
>
> Wrong.
>
> I ran some test on Linux for the effect of buffer size on file reading
> speed and the results were very interesting. I started with a buffer size
> of 32 bytes, tested file I/O throughput and increased it
> logarithmically by
> about 2% for each step.  As expected, the time to read 1mb improved as the
> buffer size increased until it hit a minimum for a buffer size of around
> 4-8k (the graph is very noisy so it's hard to tell) and then rose back up
> to a value that is 10-20% worse for buffer sizes of 32-64k and remained
> more-or-less constant for anything higher.
>
> This performance curve may be a result of the CPU cache or the OS
> architecture.
>
> The chunk size used by xreadlines is 8k which is about the optimum
> value (at least for linux).
>
> 	Oren

Disk cluster size? Fragmentation?

Dave LeBlanc
Seattle, WA USA