Threaded for loop

[skip at pobox.com]

    John> I want to do something like this:

    John> for i = 1 in range(0,N):
    John>  for j = 1 in range(0,N):
    John>    D[i][j] = calculate(i,j)

    John> I would like to now do this using a fixed number of threads, say
    John> 10 threads.  What is the easiest way to do the "parfor" in python?

I'd create a queue containing 10 tokens.  Pull a token off the queue, invoke
the thread with the parameters for its chunk, have it compute its bit, lock
D, update it, unlock it, then return the token to the token queue. Sketching
(and completely untested):

    # Calculate one row of D
    def calcrow(i, N, token, Tqueue, Dqueue):
        d = [0.0] * N
        for j in range(N):
            d[j] = calculate(i, j)
        D = Dqueue.get()
        D[i][:] = d
        Dqueue.put(D)
        Tqueue.put(token)

    # This queue limits the number of simultaneous threads
    Tqueue = Queue.Queue()
    for i in range(10):
        Tqueue.put(i)

    # This queue guards the shared matrix, D
    Dqueue = Queue.Queue()
    D = []
    for i in range(N):
        D.append([0.0] * N)
    Dqueue.put(D)

    for i in range(N):
        token = Tqueue.get()
        t = threading.Thread(target=calcrow, args=(i, N, token, Tqueue,
                                                   Dqueue))
        t.start()

Skip