Running a python farm

[Tim Churches]

Ian McConnell wrote:
> What's the pythonic way of sending out a set of requests in parallel?
> 
> My program throws an image at the server and then waits for 
> the result. I'm currently using this bit of socket code to 
> send an image to server on another machine.
> What should I be looking for, for a pythonic solution? I'm 
> stuck for terminology? Threads? Load balancing? Processors 
> farms? Any pointers or suggestions welcome.
> 
> I don't have to do this too often, so I would prefer a simple 
> solution over performance.

PyRO (see htpp://pyro.sf.net) might be useful to handle the
communications and related RPC chores. You could also use a parallel
framework (at the expense of complexity) - for example Ole Nielsen's
PyPar module, based on MPI - see
http://datamining.anu.edu.au/~ole/pypar/

Then there is the Spread framework - see
http://www.python.org/other/spread/ - and in particular lots of useful
RPC examples using Spread by Gian Paolo Ciceri at
http://pyhacks.suddenthinks.com/

Please provide feedback to this list on your experience with any of
these.

Tim C