[SciPy-user] General parallel processing question

[Andrew Straw]

Brian Granger wrote:
> There is one critical point though that in my mind makes non-blocking
> sockets absolutely necessary for this type of thing (Twisted uses
> non-blocking sockets I don't think Pyro does).  If you use blocking
> sockets a CPU bound activity in one process will *completely block*
> the execution path in the other process.  Because of this if you use
> blocking sockets your system will not experience any performance
> benefits even though it appears to be running in parallel.
>   

I just want to clear up any misconception about Pyro - there are several
ways to avoid the issue Brian mentions. The easiest are "one way" calls.
Another approach is to use multi-threading. Anyhow, I find Pyro to be a
quick and easy way to get started on IPC.

[Note: Pyro's approach is apparently inflammatory to some of the Twisted
developers, I think because they're (rightfully) concerned about
security. However, if you're running on a single, (possibly multi-core)
machine or a trusted LAN, the security issue is a non-issue.]

-Andrew