[issue33695] Have shutil.copytree(), copy() and copystat() use cached scandir() stat()s
Giampaolo Rodola'
report at bugs.python.org
Sat Jun 23 06:53:20 EDT 2018
Giampaolo Rodola' <g.rodola at gmail.com> added the comment:
PR at: https://github.com/python/cpython/pull/7874.
I re-ran benchmarks since shutil code changed after #33695. Linux went from +13.5% to 8.8% and Windows went from +17% to 20.7%.
In the PR I explicitly avoided using a context manager around os.scandir() for now so that patch it's easier to review (will add it before pushing).
Linux (+8.8%)
=============
without patch:
$ ./python bench-copytree.py
Priming the system's cache...
7956 files and dirs, repeat 1/3... min = 0.604s
7956 files and dirs, repeat 2/3... min = 0.603s
7956 files and dirs, repeat 3/3... min = 0.601s
with patch:
$ ./python bench-copytree.py
Priming the system's cache...
7956 files and dirs, repeat 1/3... min = 0.557s
7956 files and dirs, repeat 2/3... min = 0.548s
7956 files and dirs, repeat 3/3... min = 0.548s
best result = 0.548s
Windows (+20.7%)
================
without patch:
C:\Users\user\Desktop>cpython\PCbuild\win32\python.exe cpython\bench-copytree.py
Priming the system's cache...
7956 files and dirs, repeat 1/3... min = 8.275s
7956 files and dirs, repeat 2/3... min = 8.018s
7956 files and dirs, repeat 3/3... min = 7.978s
best result = 7.978s
With patch:
C:\Users\user\Desktop>cpython\PCbuild\win32\python.exe cpython\bench-copytree.py
Priming the system's cache...
7956 files and dirs, repeat 1/3... min = 6.609s
7956 files and dirs, repeat 2/3... min = 6.609s
7956 files and dirs, repeat 3/3... min = 6.609s
best result = 6.609s
----------
_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue33695>
_______________________________________
More information about the Python-bugs-list
mailing list