parallel subprocess.getoutput

Jaroslav Dobrek jaroslav.dobrek at gmail.com
Fri May 11 08:04:10 EDT 2012


Hello,

I wrote the following code for using egrep on many large files:

MY_DIR = '/my/path/to/dir'
FILES = os.listdir(MY_DIR)

def grep(regex):
    i = 0
    l = len(FILES)
    output = []
    while i < l:
        command = "egrep " + '"' + regex + '" ' + MY_DIR + '/' +
FILES[i]
        result = subprocess.getoutput(command)
        if result:
            output.append(result)
        i += 1
    return output

Yet, I don't think that the files are searched in parallel. Am I
right? How can I search them in parallel?

Jaroslav



More information about the Python-list mailing list