What sort of data structure to use?

[Peter Otten]

David Aldrich wrote:

> Hi
> 
> I have written a Python utility that performs a certain activity on some
> predefined sets of files.  Here is the outline of what I have written:
> 
> # File Set A
> pathA = 'pathA'
> fileListA = ['fileA1.txt', 'fileA2.txt']
> 
> # File Set B
> pathB = 'pathB'
> fileListB = ['fileB1.txt', 'fileB2.txt', 'fileB3.txt']
> 
> myFunc1(pathA, fileListA)
> myFunc2(pathA, fileListA)
> 
> myFunc1(pathB, fileListB)
> myFunc2(pathB, fileListB)
> 
> I want to add more file sets, so I really want to add the sets to a list
> and iterate over the list, calling myFunc1 & myFunc2 for each item.
> 
> My question is: what sort of data structure could I use to organise this,
> given that I want to associate a set of files with each path and that, for
> each set, there is an arbitrary number of files?

I'd start simple and put (path, files) pairs into a list:

path_files_pairs = [
    ("pathA", ["fileA1.txt", "fileA2.txt", ...]),
    ("pathB", ["fileB1.txt", ...]),
]

for path, files in path_files_pairs:
    func1(path, files)
    func2(path, files)

You can always add complications later:

import glob
import os

class VirtualFileset:
    def __init__(self, folder, pattern):
        self.folder = folder
        self.pattern = pattern
    def __iter__(self):
        yield self.folder
        yield glob.glob(os.path.join(self.folder, self.pattern))

path_files_pairs = [
    ("pathA", ["fileA1.txt", "fileA2.txt", ...]),
    ("pathB", ["fileB1.txt", ...]),
    VirtualFileset("pathC", "*.py"), # all python files in directory "pathC"
]

for path, files in path_files_pairs:
    func1(path, files)
    func2(path, files)