New parallel computing module for Python

[konrad.hinsen at laposte.net]

The latest development release of ScientificPython (2.7), which is
available now at

	http://sourcesup.cru.fr/projects/scientific-py/

contains a new module for parallel computing using a master-slave
model: a master process defines computational tasks, and any number
of slave processes execute them. The main advantages of this approach
are simplicity and the possibility to run a variable number of slave
processes. Slave processes can even be killed during execution; as  
long as at least one slave process remains, the computation will  
terminate correctly.

Here is a very simple example that illustrates parallel computation  
using the new module:


from Scientific.DistributedComputing.MasterSlave \
                                     import MasterProcess, SlaveProcess
from Scientific import N
import sys

class Master(MasterProcess):

     def run(self):
         for i in range(5):
             # request a computation task of type "sqrt"
             task_id = self.requestTask("sqrt", float(i))
         for i in range(5):
             # retrieve the result from a "sqrt" computation task
             task_id, tag, result = self.retrieveResult("sqrt")
             print result

class SquareRoot(SlaveProcess):

     # implement the computation task "sqrt"
     def do_sqrt(self, x):
         return (x, N.sqrt(x))

if len(sys.argv) == 2 and sys.argv[1] == "master":
     master = True
elif len(sys.argv) == 2 and sys.argv[1] == "slave":
     master = False
else:
     print "Argument must be 'master' or 'slave'"
     raise SystemExit

if master:
     process = Master("demo")
else:
     process = SquareRoot("demo")

process.start()

The communication between processes is handled by Pyro  
(pyro.sourceforge.net), which has to be installed as well.

Comments are welcome!
--
---------------------------------------------------------------------
Konrad Hinsen
Centre de Biophysique Moléculaire, CNRS Orléans
Synchrotron Soleil - Division Expériences
Saint Aubin - BP 48
91192 Gif sur Yvette Cedex, France
Tel. +33-1 69 35 97 15
E-Mail: hinsen at cnrs-orleans.fr
---------------------------------------------------------------------