Generators and nested functions

[Edward C. Jones]

Thanks. Here is a variant which also yields the original "top" directory:

def walk_gen(top):
     yield top
     if not os.path.isdir(top):
         return
     names = os.listdir(top)
     for name in names:
         fullname = os.path.join(top, name)
         for name2 in walk_gen(fullname):
             yield name2

I think a generator like this should be put in os.path. Are there other 
generators that are so useful that they need to be in official Python? 
Perhaps depth-first traversal of nested lists.

Steven Majewski wrote:

> 
> On Fri, 25 Jan 2002, Edward C. Jones wrote:
> 
> 
>>The following code would be a simple way for doing a useful thing except
>>that it doesn't work. Why not? Any easy fixes?
>>
>>from __future__ import generators
>>import os
>>
>>def walk_dirs(startdir):
>>     def visit(arg, dirname, names):
>>	yield dirname
>>
>>     os.path.walk(startdir, visit, None)
>>
>>if __name__ == '__main__':
>>     for dir in walk_dirs('/some/small/directory'):
>>	print dir
>>
>>
> 
> Replace the line 'yield dirname' with 'print dirname' and run,
> and you'll see what's going on. The visit function doesn't
> need to return a value -- it needs to have side effects.
> os.path.walk is throwing away any return values, so there's
> no way for your "yield" to get thru.
> 
> Programming with callback function always seems a bit inside-out
> to me, which is one of the reasons I like generators for this
> sort of path walking.
> 
> For a generator version, try something like:
> 
> 
>>>>def files(start):
>>>>
> ...     for x in os.listdir(start):
> ...             x = os.path.join( start, x )
> ...             yield x
> ...             if os.path.isdir( x ):
> ...                     for y in files(x): yield(y)
> ...
> 
>>>>for f in files('.') : print f
>>>>
> 
> 
> With os.path.walk, if you want to trim or filter the files, you need
> to do it in the visit function. With a generator version, you can
> filter the output of the generator. ( However, trimming branches
> that you shouldn't follow is a trickier problem! )
> 
> For another method of composing generators see:
> <http://groups.google.com/groups?q=generators+group:comp.lang.python+author:Majewski&hl=en&scoring=d&as_drrb=b&as_mind=12&as_minm=1&as_miny=2001&as_maxd=20&as_maxm=12&as_maxy=2001&rnum=1&selm=mailman.1008090197.6318.python-list%40python.org>
> 
> which lets you compose filters and generators to do something like:
> 
> for x in Files() & isGif & fileOwner('sdm7g') & fileLargerThan(512):
> 	do_something_with(x)
> 
> 
> -- Steve Majewski