Good thread pool module

[Kinsley Turner]

> There isn't a thread pool module in the standard library, but I'm sure 
> many have been written by people in the python community.
> Anyone have a favorite? Is there one particular implementation that's 
> recommended?
> 
> Not looking for anything fancy, just something that lets me queue up 
> tasks to be performed by a pool of threads and then retrieve results 
> when the tasks complete.


I wrote this one the other day, it doesn't save the results though.
You could probably just add to an output queue at the end of
threadFunction().  It was designed to handle short-running HTTP requests,
not larger jobs, this is reflected in the probablyIdle().


Use it like this:
    pool.addJob(<function>,<arg1>,<arg2>,...,<argN>)

e.g.:

def greet(times):
    for i in range(times):
        print "Hello World"

pool = ThreadPool(64)
poll.addJob(greet,3)
...




# -*- coding: iso-8859-1 -*-
import Queue
import thread

class ThreadPool:
    "A pool of worker threads, efficiently waiting for a task" 
    def __init__(self,max=0):
        self.queue         = Queue.Queue()
        self.threads       = []
        self.stopping      = False
        self.max_threads   = 16

        if (max > 0):
            self.max_threads = max

        ### Start the threads up, waiting on the empty Queue
        for i in range(self.max_threads):
 self.threads.append(thread.start_new_thread(self.threadFunction,()))

    def addJob(self,function,*args,**kwargs):
        self.queue.put((function,args,kwargs),True)

    def threadFunction(self):
        while (not self.stopping):
            task = self.queue.get(True)
            if (task != None):
                function,args,kwargs = task
                function(*args,**kwargs)
 
    def probablyIdle(self):
        return self.queue.empty()

    def getApproximateQueueSize(self):
        return self.queue.qsize()




Please consider our environment before printing this email.

WARNING - This email and any attachments may be confidential. If received in error, please delete and inform us by return email. Because emails and attachments may be interfered with, may contain computer viruses or other defects and may not be successfully replicated on other systems, you must be cautious. Westpac cannot guarantee that what you receive is what we sent. If you have any doubts about the authenticity of an email by Westpac, please contact us immediately.

It is also important to check for viruses and defects before opening or using attachments. Westpac's liability is limited to resupplying any affected attachments.

This email and its attachments are not intended to constitute any form of financial advice or recommendation of, or an offer to buy or offer to sell, any security or other financial product. We recommend that you seek your own independent legal or financial advice before proceeding with any investment decision.

Westpac Institutional Bank is a division of Westpac Banking Corporation, a company registered in New South Wales in Australia under the Corporations Act 2001 (Cth). Westpac is authorised and regulated in the United Kingdom by the Financial Services Authority and is registered at Cardiff in the United Kingdom as Branch No. BR 106. Westpac operates in the United States of America as a federally chartered branch, regulated by the Office of the Comptroller of the Currency.

Westpac Banking Corporation ABN 33 007 457 141.