Optimizing sockets to 1000 requests/sec? (Was: Sockets and messaging services)

[Stephen]

> >> IMHO the SMTP and POP protocols provide the same benefit (and there
>     >> are Python modules for those protocols).
> 
>     shriek> Using a mailserver as a messaging platform for middleware ?
> 
> It's not all that far-fetched.  I know some people have experimented with
> XML-RPC over SMTP (which is no longer XML-RPC, but that's a nit).

OK, let me show an example where I think we need some sort
of basic message queue ~ 

I've been trying to benchmark socket server and can't seem
to get it past serving 200 requests per second.

Here's the test code ~


# -----------------------------------------
# SOCKET SERVER CODE - JUST SENDS A REPLY
# -----------------------------------------

#! /usr/bin/python

import SocketServer, string, socket, time

total_requests = 0

class RequestServer(SocketServer.ThreadingTCPServer):
    allow_reuse_address = 1
    
class RequestHandler(SocketServer.StreamRequestHandler):
    def handle(self):
        global total_requests
        # ACCEPT :
        host, port = self.client_address
        request = ""
        while 1:
            line = self.rfile.readline()
            if line in (None, "\r\n", ""):
                break
            request = request + line
        request = string.rstrip(request)
        self.wfile.write("Received OK")
        total_requests = total_requests + 1
        if total_requests % 100 == 0 :
            print "%s - %s" % (total_requests, time.time())

def main():
    server = RequestServer(("127.0.0.1", 8888), RequestHandler)
    print "Listening to socket [%s:%s]" % ("127.0.0.1", 8888)
    server.serve_forever()


# -------------------------------------------
# CLIENT CODE FOR TESTING 
# -------------------------------------------

#!/usr/local/python

test_msg = "This is just a test message"

import socket, thread, time

total_sent = 0

NUMTHREADS = 20

def send_request() : 
    start = time.time()
    global total_sent
    for i in range(0,100) : 
        s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        s.connect(("127.0.0.1", 8888))
        s.send("%s\n\r\n" % test_msg)
        s.recv(512)
        s.close()
        total_sent = total_sent + 1
        if total_sent % 100 == 0 :
            print "total sent = %s" % total_sent
    end = time.time()
    print "exiting thread. duration %s" % (end - start) 

total_sent = 0
for i in range(0,NUMTHREADS) : 
    thread.start_new_thread(send_request, ())


Results ~

When NUMTHREADS = 2, 200 requests are handled in < 0.5 secs
When NUMTHREADS = 5, 500 requests are handled in 2.5 secs

But when NUMTHREADS = 20, half the threads die with this error :
Unhandled exception in thread:
Traceback (most recent call last):
  File "<stdin>", line 6, in send_request
  File "<string>", line 1, in connect
socket.error: (10061, 'Connection refused')

and it just gets worse when NUMTHREADS = 50.

I'm guessing this means that the server is max'ed out and
can't handle any more.

How should I handle this error ?  Yes, with a "try/except" 
but what should the "except" do, because if the server is
too busy, it's probably no point retrying immediately. 

So, you see, I was looking for something more immediate than
SMTP, that also takes care of the message queueing. Writing
code to manage the queue and buffering could be horrendous. 

Is there any way to get this up to say 1,000 requests/sec peak load ?

In reality, I'm going to want to put some Database query
into the RequestHandler too so it'd be even slower. 
Does this sound possible ?

Am-I-asking-too-much-of-Python-ly-your's,

Stephen