parallel processing in standard library

[Konrad Hinsen]

Emin.shopper Martinian.shopper wrote:

> Is there any hope of a parallel processing toolkit being  
> incorporated into the python standard library? I've seen a wide  
> variety of toolkits each with various features and limitations.  
> Unfortunately, each has its own API. For coarse-grained  
> parallelism, I suspect I'd be pretty happy with many of the  
> existing toolkits, but if I'm going to pick one API to learn and  
> program to, I'd rather pick one that I'm confident is going to be  
> supported for a while.

I don't think that parallel computing is mature enough to allow the  
standardization of APIs, except within a given and well specified  
parallel computing model such as message passing.

The Python Wiki has an impressive list of parallel processing options  
for Python (see http://wiki.python.org/moin/ParallelProcessing). With  
the exception of the various MPI interfaces, I don't think that any  
two of them are based on the same parallel computing model. I don't  
expect this situation to change any time soon, as parallel computing  
is still very much experimental. Whereas sequential computing has  
well-tested software engineering techniques, reliable libraries that  
can be combined into programs, and ever improving testing techniques,  
none of these exist for parallel computing.

For an overview of parallel computing models and for a more detailed  
description of one of them as implemented in Python, please see my  
recent article in "Computing in Science and Engineering":

http://www.computer.org/portal/site/cise/index.jsp? 
pageID=cise_level1&path=cise/2007/n6&file=sci.xml&xsl=article.xsl

Konrad.