[Python-Dev] CHUNKSIZE

[Martin v. Loewis]

Michael Gilfix <mgilfix@eecs.tufts.edu> writes:

>   This comment is more of a question.. Does the chunksize not have
> to do with how the filesystem stores blocks? 

Yes, but probably in non-obvious ways. Many file systems attempt to
allocate subsequent blocks in a file into subsequent sectors of the
disk. As a result, you can get lots of data in a single rotation of
the disk. *If* you are going to use all the data, it is better to
transfer them all in one chunk - the kernel needs to setup IO only for
a single operation, which then completes at the burst speed of the
device. If the disk repositions heads between reads, performance goes
down.

Regards,
Martin