using queue

Tim Arnold tim.arnold at sas.com
Tue Sep 1 13:53:10 EDT 2009


Hi, I've been using the threading module with each thread as a key in a 
dictionary. I've been reading about Queues though and it looks like that's 
what I should be using instead. Just checking here to see if I'm on the 
right path.
The code I have currently compiles a bunch of chapters in a book (no more 
than 80 jobs at a time) and just waits for them all to finish:

        max_running = 80
        threads = dict()
        current = 1
        chaps = [x.config['name'] for x in self.document.chapter_objects]
        while current <= len(chaps):
            running = len([x for x in threads.keys() if 
threads[x].isAlive()])
            if running == max_running:
                time.sleep(10)
            else:
                chap = chaps[current - 1]
                c = self.compiler(self.document.config['name'], chap)
                threads[chap] = threading.Thread(target=c.compile)
                threads[chap].start()
                current += 1

        for thread in threads.keys():
            threads[thread].join(3600.0)
---------------------------------
but I think Queue could do a lot of the above work for me. Here is 
pseudocode for what I'm thinking:

q = Queue(maxsize=80)
for chap in [x.config['name'] for x in self.document.chapter_objects]:
    c = self.compiler(self.document.config['name'], chap)
    t = threading.Thread(target=c.compile)
    t.start()
    q.put(t)
q.join()

is that the right idea?
thanks,
--Tim Arnold





More information about the Python-list mailing list