[issue25156] shutil.copyfile should internally use os.sendfile when possible

[desbma]

desbma added the comment:

Thoughts anyone?
Here is a patch that implements the change.

My tests show a 30-40% performance improvement for 128KB-512MB single file copy:

128 KB file copy:

$ dd if=/dev/urandom of=/tmp/f1 bs=1K count=128

Without the patch:
$ ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'
10000 loops, best of 3: 109 usec per loop

With the patch:
$ ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'
10000 loops, best of 3: 75.7 usec per loop

--------
8 MB file copy:

$ dd if=/dev/urandom of=/tmp/f1 bs=1M count=8

Without the patch:
$ ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'
100 loops, best of 3: 4.99 msec per loop

With the patch:
$ ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'
100 loops, best of 3: 3.03 msec per loop

--------
512 MB file copy:

$ dd if=/dev/urandom of=/tmp/f1 bs=1M count=512

Without the patch:
$ ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'
10 loops, best of 3: 305 msec per loop

With the patch:
$ ./python -m timeit -s 'import shutil; p1 = "/tmp/f1"; p2 = "/tmp/f2"' 'shutil.copyfile(p1, p2)'
10 loops, best of 3: 178 msec per loop

----------
keywords: +patch
Added file: http://bugs.python.org/file40881/issue25156.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25156>
_______________________________________