Network performance

[Sion Arrowsmith]

Roland Hedberg  <roland.hedberg at adm.umu.se> wrote:
> [ ... ]
>The client sends a number of lines (each ending with \n) and ends one  
>set of lines with a empty line.
>When the client sends a line with only a "." it means "I'm done close  
>the connection".
>
>Letting the client open the connection and sending a number of sets I  
>was surprised to find that the performance was equal to what Twisted/ 
>XMLRPC did. Around 200 ms per set, not significantly less.
>
>So what can be done to speed up things or is Python incapable of  
>supporting fast networking (just a teaser).

That 200ms is a dead giveaway (if you've run into this issue before).
If you've got a Linux box to hand, try sticking the server on there
and see what happens.

The key to what is going on is here:
http://www.port80software.com/200ok/archive/2005/01/31/317.aspx

The easy solutions are to either change:

>	def send( self, rdf ):
>		self.s.send( rdf )
>		self.s.send( "\n" )

to

	def send( self, rdf ):
		self.s.send( rdf+"\n" )

or to self.s.setsockopt(socket.SOL_TCP, socket.TCP_NODELAY, 0) after
connecting self.s .

What's going on is something like this:

1. Client sends rdf

2. Server receives rdf. But it hasn't received the \n yet, so it doesn't
have a response to send yet. And because of delayed ACK, it won't send
the ACK without the DATA packet containing that response.

3. Meanwhile, the client tries to send \n, but because Nagle is on, it
would make an underful packet, and it hasn't received the ACK from the
last packet yet, so it holds on to the \n.

4. Eventually (after 200ms on Windows and, I believe, BSDs, hence OSX,
but much less on Linux) the Server gives up waiting for a DATA packet
and sends the ACK anyway.

5. Client gets this ACK and sends the \n.

6. At which point the Server has a complete line it can process, and
sends its reponse in a DATA with the ACK to the \n "piggybacked".

So the solutions are either send rdf+"\n" as a single packet in 1, and
the exchange will skip straight to 6, avoiding the 200ms delay; or
disable Nagle so 3 doesn't hold and the Client sends the \n
immediately instead of waiting for the ACK in 5.

-- 
\S -- siona at chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
  ___  |  "Frankly I have no feelings towards penguins one way or the other"
  \X/  |    -- Arthur C. Clarke
   her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump