Optimizing sockets to 1000 requests/sec? (Was: Sockets and messaging services)

[Gordon McMillan]

Stephen wrote:

[Gordon]
>> About 3 years ago, I wrote a Python message-slinger as a drop-in
>> replacement for my client's C code. It did around 600 msgs / sec (vs
>> <200 / sec for the client's code). That was Python 1.4 on 3 year old
>> hardware. 
> 
> Thanks for the example, Gordon. Gives me the confidence knowing that
> "it is possible" ~ now I just have to work out how. It's very
> reassuring, rather than having doubts and asking myself "should I 
> be doing this directly in C/Java instead?" whenever I hit a problem.  
> 
>> So 1000 / sec is quite possible, but you'll never get there with
>> blocking sockets. 
> 
> Even if they're threaded ?  

Even if they're threaded. Thread creation is high-overhead (allocating megs 
of memory, for one thing). If you do use threads, it's better to create a 
thread pool and reuse them. Depends on platform, but if you're using more 
than (say) a dozen threads, you're probably wasting resources. It's also 
vastly more efficient to use select than to have the OS cycle through a 
bunch of context-switches, testing for socket activity. 

Basically, the "one thread per socket" model sucks.
 
> May I ask what you used for the 600 msgs/sec ? Select's and asyncore ?

It was very similar to asyncore.
 
- Gordon