fork() and wait()

Chuck May NmOaSyPcAM at imsweb.com
Mon Oct 14 15:13:06 EDT 2002


I am trying to write an application that forks off processes and waits 
for everything to complete before doing other calculations.  The one 
catch is that I would like to limit the number of concurrent child 
processes.  In other words, let's say I have 10 "jobs" that need to be 
done, but I'd like no more than 4 running at the same time.  Here is 
what I cane up with:

#!/usr/bin/env python

import os
import sys
import random
from time import *

num_children = 0
max_children = 4
jobs         = range(10)

def runJob(job_num, sleep_time):
    print "   START: Job #%d (pid %d) with sleep(%d)" % (job_num, 
os.getpid(), sleep_time)
    sleep(sleep_time)
    print "   END:   Job #%d (pid %d)" % (job_num, os.getpid())
    sys.exit(0)

while len(jobs) > 0 or num_children > 0:
    if len(jobs) > 0:
        sleep_time = random.randrange(1,6)
        job_num = jobs.pop(0)
        print "FORK: Job #", job_num
        result = os.fork()
        if not result:  # in the child process
            runJob(job_num, sleep_time)
        else:
            num_children += 1
        
    if len(jobs) == 0 or num_children >= max_children:
        result = os.wait()
        num_children -= 1
    
print "ALL FINISHED"

It appears that this code works, but I am concerned about the timing.  
In the main loop, I am concerned that a job would finish before the 
os.wait() call.  Does anyone know if there is a "safer" method of 
achieving this goal?

Thank for any help,

--
Chuck May
IMS, Inc.
NmOaSyPcAM at imsweb.com



More information about the Python-list mailing list