Limit number of concurrent threads

[Chuck May]

I am writing a database utility that needs to run a particular process 
on every table in the database.  Currently, it is something like this:

    def do_table(table):
        # handle table operations here

    for table in all_tables:
        do_table(table)

    # summarize results

Since some tables take much longer than others, and since the table are 
not in contention with each other, I'd like to thread the operation.  
What I would ideally like is to have a set number of threads, say 10, 
running at the same time (my database has a concurrent connection 
limit).  Since there are about 80 tables, once there are 10 threads 
running I'd like to add another one as soon as one finishes.  Then I 
need to wait at the end until all the threads finish, so that I can 
summarize results.  One other nice feature would be to exit the 
application if any thread fails.

I've searching for examples that fit my problem, and I haven't found 
any.  Most examples I've found look something like this:

    threads = []
    for i in range(number_of_tasks):
        thread.append(thread_for_task)
    
    # let the threads run - just wait for shutdown
    for t in threads:
        t.join() 

which whould work fine if I wanted to start all 80 threads at once.

I've read about the Queue, but I'm not sure if it's what I want, and I 
can't seem to find any good examples on it.

Any help would be appreciated.  Thanks.

-- 
Chuck May
IMS, Inc.