Optimizing tips for os.listdir

G. S. Hayes sjdevnull at yahoo.com
Mon Sep 27 18:32:10 EDT 2004


Nick Craig-Wood <nick at craig-wood.com> wrote in message news:<slrnclg6vb.rjj.nick at irishsea.home.craig-wood.com>...
> Thomas <2002 at weholt.org> wrote:
> >  [os.path.join(path, p) for p in os.listdir(path) if \
> >  os.path.isdir(os.path.join(path, p))]
> > 
> >  to get a list of folders in a given directory, skipping all plain
> >  files. When used on folders with lots of files,  it takes rather long
> >  time to finish. Just doing  a listdir, filtering out all plain files
> >  and a couple of joins, I didn't think this would take so long. 
> How many files, what OS and what filing system?
> 
> Under a unix based OS the above will translate to 1
> opendir()/readdir()/closedir() and 1 stat() for each file.  There
> isn't a quicker way in terms of system calls AFAIK.

Under Linux, readdir() returns a struct dirent that has a d_type
member indicating the file type (DT_DIR for directories) so you can
avoid calling stat() on each file.  I thought some BSD systems did
this as well.

I don't see how to get at this information from Python without making
the extra syscalls.



More information about the Python-list mailing list