[issue11406] There is no os.listdir() equivalent returning generator instead of list

[Gregory P. Smith]

Gregory P. Smith added the comment:

Your objection is noted but it is wrong.

A Python program today cannot process arbitrarily large directories within a fixed amount of ram today due to os.listdir. This makes it unsuitable for file system cleanup tasks that we have run into on production servers.  This fixes that without requiring an extension module or fragile ctypes code to work around the deficiency.

It _would've been nice_ for os.listdir to be updated to be an iterator with 3.0 but it wasn't... so we're left supporting its legacy interface rather than just replacing it with this.

If think this functionality belongs in a module other than os, please suggest where.

long term: os.walk and os.fwalk are also unusable on arbitrary filesystems with large directories for the same reason because they use os.listdir.  providing a non-directory-preloading version of those is outside the scope of this issue.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue11406>
_______________________________________