Python threading?

[Mark Hammond]

Aahz wrote:
> In article <f9Pi9.10275$Lg2.2191294 at news2.news.adelphia.net>,
> Robert Oschler <Oschler at earthlink.net> wrote:
> 
>>Also, what is a reasonable number of threads to expect to be able to run
>>before context switching overhead becomes a problem (I'm using a PIII 500
>>Mhz with 512MB ram if that helps).
> 
> 
> It depends.  For pure Python-based computation, more than one thread
> will cost you more than you gain.  If it's almost pure I/O (file or
> socket stuff), you might be able to use as many as a couple of hundred
> threads.  Mostly, you're probably best off somewhere between five and
> thirty threads, depending on what you're doing.

I would have thought that a brave call.  If you are performing lots of 
IO and need many many connections, you are almost certainly better off 
using an "asynchronous" approach, much like asyncore.  With a large 
number of threads (where "large" can be as low as single figures) the OS 
overhead of switching will be greater than your internal overhead of a 
more complicated design.

A "one thread per connection" model is doomed from the start - 
therefore, in most applications, I would consider 5 threads a large 
number - particularly when all are actively doing something.  OTOH, if 
you are willing to accept that your first version will accept a max of 
~20 connections, the brain-power saved by using threads beats the 
additional OS overhead by a mile ;)

Mark.