[ANN] dispy: distribute computations and execute in parallel

[Giridhar Pemmasani]

Hello,

I would like to announce dispy (http://dispy.sf.net) that can
distribute and parallelize computations among computing nodes over
network (yes, yet another implementation of parallelization). This is
useful for problems in SIMD paradigm where a computation can be
executed with multiple data simultaneously. Salient features of dispy
are:

 * Computations (Python functions or standalone programs) and its
   dependencies (files, Python functions, classes, modules) are
   distributed automatically as and when needed.

 * Computation nodes can be anywhere on the network. For security,
   either simple hash based authentication or SSL encryption can be
   used.

 * A computation may specify which nodes are allowed to execute it
   (for now, using simple patterns of IP addresses).

 * After each execution is finished, the results of execution, output,
   errors and exception trace are made available for further
   processing.

 * Nodes may become available dynamically: dispy will schedule jobs
   whenever a node is available and computations can use that node. If
   a node fails while executing scheduled jobs, those jobs may be
   resubmitted to other nodes if computations allow it.

 * dispy can be used in a single process to use all the nodes
   exclusively (simpler to use) or in multiple processes sharing the
   nodes.

Currently dispy works with Python 2.7 and has been tested with nodes
running Linux and OS X.

Thanks,
Giri
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110910/69788e3c/attachment.html>