Iterating over files of a huge directory

[Oscar Benjamin]

On 17 December 2012 18:40, Evan Driscoll <driscoll at cs.wisc.edu> wrote:
> On 12/17/2012 09:52 AM, Oscar Benjamin wrote:
>> https://github.com/benhoyt/betterwalk
>
> This is very useful to know about; thanks.
>
> I actually wrote something very similar on my own (I wanted to get
> information about whether each directory entry was a file, directory,
> symlink, etc. without separate stat() calls).

The initial goal of betterwalk seemed to be the ability to do os.walk
with fewer stat calls. I think the information you want is part of
what betterwalk finds "for free" from the underlying OS iteration
(without the need to call stat()) but I'm not sure.

> (Also just for the record and anyone looking for other posts, I'd guess
> said discussion was on Python-dev. I don't look at even remotely
> everything on python-list (there's just too much), but I do skim most
> subject lines and I haven't noticed any discussion on it before now.)

Actually, it was python-ideas:
http://thread.gmane.org/gmane.comp.python.ideas/17932
http://thread.gmane.org/gmane.comp.python.ideas/17757