Dealing with paths (Unix question)

[Bjorn Pettersen]

> From: paul [mailto:appleseed-cast~NOSPAM~@breathe.com] 
> 
> hey there all!
> i'm writing some little programs, and i'd like to know about 
> how to deal with directories on unix.  i've read up on this, 
> but the docs don't really mention much on it.  do i use the 
> pickle module, or something?

If you could write up a bugreport on
http://sourceforge.net/tracker/?group_id=5470&atid=105470 including what
you were looking for and where you expected it to be it has a much
better chance of getting fixed (and your input is probably more valuable
than someone that allready knows about paths in Python :-)

> what i'd like to do is read in a path, say
> /home/username/images/
> and that path contains a dir called downloads and some png files, 
> and it also contains another dir called icons, which holds 
> jpg, png and gif files. Now, is it posible to go through that 
> top directory and recursively go through /icons/ and /downloads/ 
> and pick out all files of a certain extension(only png files, 
> for example)?
> 
> something like:
> for each_file in /home/username/images:
> 	if image_extension is png:
> 		add image path to list
> but as i said, i need something to work recursively though an 
> arbitrary number of directories.
[...]

Well, you'll have a problem pruning the search space if you don't know
where in the hierarchy they can be. My solution (below) is cheating by
not doing any pruning except in directories containin either 'icons' or
'downloads', and then it prunes every thing except those... It's also
slightly overambitious in that you can match anything, not just
extensions (I needed a replacement for Lib/lib-old/find.py, so...)  This
uses the new os.walk, which is much easier to explain than the old
os.path.walk():

import os
import fnmatch # much flexibility for little pain :-)

def findFiles(patterns, startdir=os.curdir, modifySearchPath=lambda
x:None):
    """patterns: sequence of regular expressions (shell style, not re)
defining a 
                 match. See fnmatch documentation for the specifics.
       startdir: ...
       modifySearchPath: callable taking a list of directories and
modifiying/pruning
                         it *in place* (in particular any return value
from this
                         callable is discarded :-)
    """
    for root, dirs, files in os.walk(startdir): 
        modifySearchPath(dirs)
 
        for pattern in patterns:
            files = fnmatch.filter(files, pattern)
            for file in files:
                # use abspath on windows to get the drive included when
                # startdir is os.curdir (I'm assuming it doesn't do any
                # harm on *nix, but I don't have a box here to test..)
                yield os.path.abspath(os.path.join(root, file))

# define the pruning algorithm, this is probably not exactly what you 
# want (see note above) -- season to taste.
def myDirFilter(directories):
    icons_p = 'icons' in directories
    downloads_p = 'downloads' in directories
    if not icons_p or downloads_p:
        # this level doesn't have anything we're looking for so 
        # don't modify directories.
        return
    directories[:] = [] # note: slice assignment to do *in place*
remove-all.
    # add back the only directories we're interested in.
    if icons_p: directories.append('icons')
    if downloads_p: directories.append('downloads')

# define what we're looking for
imageFiles = ('*.jpe?g', '*.gif', '*.png')

# loop and enjoy.
for filename in findFiles(patterns=imageFiles,
modifySearchPath=myDirFilter):
    print filename

hth,
-- bjorn