Optimizing sockets to 1000 requests/sec? (Was: Sockets and messaging services)

[Scherer, Bill]

You've got a multithreaded client hitting a single threaded, 
synchronous server.  You can do significantly better if you 
thread your server, or use asyncore (or a combination of the 
two).

On 14 Nov 2001, Stephen wrote:

> > >> IMHO the SMTP and POP protocols provide the same benefit (and there
> >     >> are Python modules for those protocols).
> > 
> >     shriek> Using a mailserver as a messaging platform for middleware ?
> > 
> > It's not all that far-fetched.  I know some people have experimented with
> > XML-RPC over SMTP (which is no longer XML-RPC, but that's a nit).
> 
> OK, let me show an example where I think we need some sort
> of basic message queue ~ 
> 
> I've been trying to benchmark socket server and can't seem
> to get it past serving 200 requests per second.
> 
> Here's the test code ~
> 
> 
> # -----------------------------------------
> # SOCKET SERVER CODE - JUST SENDS A REPLY
> # -----------------------------------------
> 
> #! /usr/bin/python
> 
> import SocketServer, string, socket, time
> 
> total_requests = 0
> 
> class RequestServer(SocketServer.ThreadingTCPServer):
>     allow_reuse_address = 1
>     
> class RequestHandler(SocketServer.StreamRequestHandler):
>     def handle(self):
>         global total_requests
>         # ACCEPT :
>         host, port = self.client_address
>         request = ""
>         while 1:
>             line = self.rfile.readline()
>             if line in (None, "\r\n", ""):
>                 break
>             request = request + line
>         request = string.rstrip(request)
>         self.wfile.write("Received OK")
>         total_requests = total_requests + 1
>         if total_requests % 100 == 0 :
>             print "%s - %s" % (total_requests, time.time())
> 
> def main():
>     server = RequestServer(("127.0.0.1", 8888), RequestHandler)
>     print "Listening to socket [%s:%s]" % ("127.0.0.1", 8888)
>     server.serve_forever()
> 
> 
> # -------------------------------------------
> # CLIENT CODE FOR TESTING 
> # -------------------------------------------
> 
> #!/usr/local/python
> 
> test_msg = "This is just a test message"
> 
> import socket, thread, time
> 
> total_sent = 0
> 
> NUMTHREADS = 20
> 
> def send_request() : 
>     start = time.time()
>     global total_sent
>     for i in range(0,100) : 
>         s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>         s.connect(("127.0.0.1", 8888))
>         s.send("%s\n\r\n" % test_msg)
>         s.recv(512)
>         s.close()
>         total_sent = total_sent + 1
>         if total_sent % 100 == 0 :
>             print "total sent = %s" % total_sent
>     end = time.time()
>     print "exiting thread. duration %s" % (end - start) 
> 
> total_sent = 0
> for i in range(0,NUMTHREADS) : 
>     thread.start_new_thread(send_request, ())
> 
> 
> Results ~
> 
> When NUMTHREADS = 2, 200 requests are handled in < 0.5 secs
> When NUMTHREADS = 5, 500 requests are handled in 2.5 secs
> 
> But when NUMTHREADS = 20, half the threads die with this error :
> Unhandled exception in thread:
> Traceback (most recent call last):
>   File "<stdin>", line 6, in send_request
>   File "<string>", line 1, in connect
> socket.error: (10061, 'Connection refused')
> 
> and it just gets worse when NUMTHREADS = 50.
> 
> I'm guessing this means that the server is max'ed out and
> can't handle any more.
> 
> How should I handle this error ?  Yes, with a "try/except" 
> but what should the "except" do, because if the server is
> too busy, it's probably no point retrying immediately. 
> 
> So, you see, I was looking for something more immediate than
> SMTP, that also takes care of the message queueing. Writing
> code to manage the queue and buffering could be horrendous. 
> 
> Is there any way to get this up to say 1,000 requests/sec peak load ?
> 
> In reality, I'm going to want to put some Database query
> into the RequestHandler too so it'd be even slower. 
> Does this sound possible ?
> 
> Am-I-asking-too-much-of-Python-ly-your's,
> 
> Stephen
> 

-- 
Bill.Scherer at Verizon Wireless
RHCE 807101044903581