[Python-Dev] os.walk() is going to be *fast* with scandir

[Nick Coghlan]

On 10 August 2014 13:20, Antoine Pitrou <antoine at python.org> wrote:
> Le 09/08/2014 12:43, Ben Hoyt a écrit :
>
>> Just thought I'd share some of my excitement about how fast the all-C
>> version [1] of os.scandir() is turning out to be.
>>
>> Below are the results of my scandir / walk benchmark run with three
>> different versions. I'm using an SSD, which seems to make it
>> especially faster than listdir / walk. Note that benchmark results can
>> vary a lot, depending on operating system, file system, hard drive
>> type, and the OS's caching state.
>>
>> Anyway, os.walk() can be FIFTY times as fast using os.scandir().
>
>
> Very nice results, thank you :-)

Indeed!

This may actually motivate me to start working on a redesign of
walkdir at some point, with scandir and DirEntry objects as the basis.
My original approach was just too slow to be useful in practice (at
least when working with trees on the scale of a full Fedora or RHEL
build hosted on an NFS share).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia