reading directory entries one by one

[holger krekel]

Andrew Dalke wrote:
> Michael P. Soulier:
> >    I have a few with 4000+ files in them. There's enough memory on the box
> to
> >easily handle this, but it's still not efficient.
> 
> "efficient"?  What does that mean in this context?  There are three
> metrics I can think of:
>  - development time, but a list is far easier to understand and use than
>      an iterator.  Eg, to sort given an iterator you need to put it in
>      a list() first.

right, but if your application  only *iterates* over entries then there
is nothing to understand from the user-side (except for ill cases :-).

>  - memory size, for 4000+ files at, oh, 40 bytes per name/string is
>      160K.  Which is about 1/10 of what Python is using, and less than
>      1/1000th of what most machines have.

right.
 
>  - run time, it's likely faster for Python to build one list in a
>      go than have the iterator overhead

there is currently no way to time that. 

What i also appreciate about generators/iterators is that 
they  generally decrease the *latency* for getting the first entry. 

    holger