Testing threading

[George Sakkis]

How one goes on testing a threaded program, apart from doing a few
successful runs and crossing his fingers that it at least follows the
'correct 99.9999% of the time' rule ? I've written a stripped-down
python version of Doug Lea's PooledExecutor thread pool class
(http://gee.cs.oswego.edu/dl/classes/EDU/oswego/cs/dl/util/concurrent/PooledExecutor.html),
I ran a few simple examples that seem to work as expected, but I still
wouldn't bet any serious amount on its correctness. Any general, or
specific to this problem, tesing guidelines ?

As for the 'stripped-down version', it refers to the lack of shutting
down facilities. These are based on interrupting a thread and AFAIK
this is not possible in python, is it ?

On the other hand, in addition to the main execute(command) method of
the original PooledExecutor, I added a new dispatch(callables)
generator method. This takes an iterable over zero-argument callables,
it dispatches them to separate threads according to the thread pool's
configuration (minPoolSize,maxPoolSize,etc.), collects the results in a
queue as they are computed and yields them back to the caller. Since
the order of the computed results is not in general the same with the
order of the input, the yielded items are tuples (i,result), where i is
the index of the respective callable that returned this result. As a
toy example:

from math import sqrt
from itertools import count
tasks = (lambda x=n: sqrt(x) for n in count())
pool = PooledExecutor(minPoolSize=4)
for n,s in pool.dispatch(tasks):
    print "The square root of %d is %s" % (n,s)


George