Slow network reading?

Ivan Voras ivoras at __-fer.hr-__
Wed May 10 21:57:05 EDT 2006


I have a simple network protocol client (it's a part of this: 
http://sqlcached.sourceforge.net) implemented in Python, PHP and C. 
Everything's fine, except that the Python implementation is the slowest 
- up to 30% slower than the PHP version (which implements exactly the 
same logic, in a class).

In typical usage (also in the benchmark), an object is created and 
.query is called repeatedly. Typical numbers for the benchmark are:

For Python version:

Timing 100000 INSERTs...
5964.4 qps
Timing 100000 SELECTs...
7491.0 qps

For PHP version:

Timing 100000 inserts...
7820.2 qps
Timing 100000 selects...
9926.2 qps


The main part of the client class is:

----

import os, socket, re

class SQLCacheD_Exception(Exception):
     pass

class SQLCacheD:

     DEFAULT_UNIX_SOCKET = '/tmp/sqlcached.sock'
     SC_VER_SIG = 'sqlcached-1'
     SOCK_UNIX = 'unix'
     SOCK_TCP = 'tcp'

     re_rec = re.compile(r"\+REC (\d+), (\d+)")
     re_ok = re.compile(r"\+OK (.+)")
     re_ver = re.compile(r"\+VER (.+)")

     def __init__(self, host = '/tmp/sqlcached.sock', type = 'unix'):
         if type != SQLCacheD.SOCK_UNIX:
             raise

         self.sock = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
         self.sock.connect(host)
         self.sf = self.sock.makefile('U', 4000)

         self.sf.write("VER %s\r\n" % SQLCacheD.SC_VER_SIG)
         self.sf.flush()
         if self.sf.readline().rstrip() != '+VER %s' % SQLCacheD.SC_VER_SIG:
             raise SQLCacheD_Exception("Handshake failure (invalid 
version signature?)")


     def query(self, sql):
         self.sf.write("SQL %s\r\n" % sql)
         self.sf.flush()
         resp = self.sf.readline().rstrip()
         m = SQLCacheD.re_rec.match(resp)
         if m != None: # only if some rows are returned (SELECT)
             n_rows = int(m.group(1))
             n_cols = int(m.group(2))
             cols = []
             for c in xrange(n_cols):
                 cols.append(self.sf.readline().rstrip())
             rs = []
             for r in xrange(n_rows):
                 row = {}
                 for c in cols:
                     row[c] = self.sf.readline().rstrip()
                 rs.append(row)
             return rs
         m = SQLCacheD.re_ok.match(resp)
         if m != None: # no rows returned (e.g. INSERT/UPDATE/DELETE)
             return True
         raise SQLCacheD_Exception(resp)

----

My question is: Am I missing something obvious? The C implementation is 
(as expected) the fastest with result of 10000:15000, but somehow I 
expected the Python one to be closer to, or even faster than PHP.

I tried using 'r' mode for .makefile() but it had no significant effect.



More information about the Python-list mailing list