Iterating over files of a huge directory
Gilles Lenfant
gilles.lenfant at gmail.com
Mon Dec 17 11:06:28 EST 2012
Le lundi 17 décembre 2012 16:52:19 UTC+1, Oscar Benjamin a écrit :
> On 17 December 2012 15:28, Gilles Lenfant <...> wrote:
>
>
> In the last couple of months there has been a lot of discussion (on
>
> python-list or python-dev - not sure) about creating a library to more
>
> efficiently iterate over the files in a directory. The result so far
>
> is this library on github:
>
> https://github.com/benhoyt/betterwalk
>
>
>
> It says there that
>
> """
>
> Somewhat relatedly, many people have also asked for a version of
>
> os.listdir() that yields filenames as it iterates instead of returning
>
> them as one big list.
>
>
>
> So as well as a faster walk(), BetterWalk adds iterdir_stat() and
>
> iterdir(). They're pretty easy to use, but see below for the full API
>
> docs.
>
> """
>
>
>
> Does that code work for you? If so, I imagine the author would be
>
> interested to get some feedback on how well it works.
>
>
>
> Alternatively, perhaps consider calling an external utility.
>
Many thanks for this pointer Oscar.
"betterwalk" is exactly what I was looking for. More particularly iterdir(...) and iterdir_stat(...)
I'll get a deeper look at betterwalk and provide (hopefully successful) feedback.
Cheers
--
Gilles Lenfant
More information about the Python-list
mailing list