Tips for load balancing multiple Python apps on dual/quad core processors?

[Jeffrey Froman]

Malcolm Greene wrote:

> I'm looking for tips on how to load balance running multiple Python
> applications in multi-CPU environments. My understanding is that Python
> applications and their threads are limited to a specific CPU.
> 
> Background: I have a Python utility that processes email messages. I
> suspect there's a lot of idle time while this utility waits on a remote
> email server.

While it's true that python's threads are limited to using a single CPU at a
time, it sounds like your bottleneck is network IO, not CPU power.

I imagine that your one CPU is doing a lot of idling, thumb-twiddling, etc.,
while waiting for the remote email server to respond. In such a scenario,
python's threads will work fine to speed your application. On the other
hand, adding CPUs to an IO-bound application isn't likely to speed it up at
all.


Jeffrey