Socket Speed

[Dave Brueck]

mvanaswegen at gmail.com wrote:
> Hi All
> 
> I'm busy writing a python p2p program and would like some advice.
> 
> I'm pushing about 300k/s and would like to know if there are any python
> tricks I could pull to speed things up. I'm thinking about unrolling
> some of the loops and recuding calls to my custom socket class and
> just calling recv and send directly. What is the overhead for using
> try and exception blocks ? any recomendations for recv/send sizes ?
> On the server side I'm using SocketServer. I was planning to use
> twisted
> but I don't have the time just get to get into it, SocketServer does
> the job.
> 
> I'll post some code soon.

Yeah - do post some code as 300k/s is probably *way* below the potential. In one 
project I'm working on, I do simple file transfers over a single socket with 
Python on both ends and I routinely get above 500Mbps (often into the high 
700's/low 800's - pretty close to what I could hope for for gig over copper with 
TCP/IP) and the CPU utilization is way low - I assume disk on one end or the 
other of the connection is a bottleneck now.

I use 64k send/receive buffers - above that it doesn't seem to help much for 
what I'm doing.

In any case, before you make any local optimizations (loop unrolling, etc.), I'd 
do some more investigation to find the problem because something major is 
broken, and fixing it will probably do way more to improve performance than any 
optimization "tricks".

-Dave