Parallel Python

[robert]

Paul Boddie wrote:
> parallelpyt... at gmail.com wrote:
>> The main difference between MPI python solutions and ppsmp is that with
>> MPI you have to organize both computations
>> {MPI_Comm_rank(MPI_COMM_WORLD, &id); if id==1 then ... else ....} and
>> data distribution (MPI_Send / MPI_Recv) by yourself. While with ppsmp
>> you just submit a function with arguments to the execution server and
>> retrieve the results later.
> 
> Couldn't you just provide similar conveniences on top of MPI? Searching
> for "Python MPI" yields a lot of existing work (as does "Python PVM"),
> so perhaps someone has already done so. Also, what about various grid
> toolkits?
> 
> [...]
> 
>> Overall ppsmp is still work in progress and there are other interesting
>> features which I would like to implement. This is the main reason why I
>> do not open the source of ppsmp - to have better control of its future
>> development, as advised here: http://en.wikipedia.org/wiki/Freeware :-)
> 
> Despite various probable reactions from people who will claim that
> they're comfortable with binary-only products from a single vendor, I
> think more people would be inclined to look at your software if you did
> distribute the source code, even if they then disregarded what you've
> done. My own experience with regard to releasing software is that even
> with an open source licence, most people are likely to ignore your
> projects than to suddenly jump on board and take control, and even if
> your project somehow struck a chord and attracted a lot of interested
> developers, would it really be such a bad thing? Many developers have
> different experiences and insights which can only make your project
> better, anyway.
> 
> Related to your work, I've released a parallel execution solution
> called parallel/pprocess [1] under the LGPL and haven't really heard
> about anyone really doing anything with it, let alone forking it and
> showing my original efforts in a bad light. Perhaps most of the
> downloaders believe me to be barking up the wrong tree (or just
> barking) with the approach I've taken, but I think the best thing is to
> abandon any fears of not doing things the best possible way and just be
> open to improvements and suggestions.
> 
> Paul
> 
> [1] http://www.python.org/pypi/parallel

I'd be interested in an overview.
For ease of use a major criterion for me would be a pure python 
solution, which also does the job of starting and controlling the 
other process(es) automatically right (by default) on common 
platforms.
Which of the existing (RPC) solutions are that nice?


Robert