Multithreaded Telnet sessions

[Josiah Carlson]

Richard Bird wrote:

> Thanks so much for your helpful sample. It proved to be just the nudge
> I needed to get the ball rolling.
> I've modified the old script to support threading using your outline
> and have begun to implement queueing to log information.
> 
> I can't get over how much faster it is to push a change to 800+
> routers using 20+ threads!
> 
> Anything over 20 threads seems to take processor utilization to 100%.
> Is there any benefit to starting additional threads once the CPU is at
> 100% ?

Sorry I didn't get into this earlier (it's been a while since I browsed 
through c.l.py), but I notice that you are doing socket programming with 
threads.

At one time I did use threading for sockets (senior undergrad project), 
but one thing I discovered is that threading sometimes doesn't scale (it 
didn't when it came to the sockets and protocols I was using).

On the other hand, if you can change the way you think, and spend enough 
time working with asyncore (and derivatives), you can make asynchronous 
sockets work for you*.

Depending on how much time and energy you have to spend on this, you may 
want to keep using threads and call it good.  Heck, if I didn't need so 
many fast connections, I'd have likely stuck with threads.

  - Josiah

* Right now I've got some code for a contract that is able to handle 500 
sockets (the compile-time limit is 512, and I like to leave a little 
breathing room) at 12 megs/second (on a single-processor mobile celeron 
1.3 ghz, server and client running on the same machine), using a very 
chatty line-based protocol (similar to telnet, most 'lines' are 80 bytes 
or less) and my own derivative of asyncore (I may release a generalized 
variant, after the contract has ended, if the company says it is alright).