[New-bugs-announce] [issue4336] Fix performance issues in xmlrpclib

[Kristján Valur Jónsson]

New submission from Kristján Valur Jónsson <kristjan at ccpgames.com>:

There are two performance problems in xmlrpclib.py:
POST requests use two send() calls, one to send the headers and one to 
send the data.  This can invoke the Nagle/Delayed ACK performance 
problem.  On many socket implementations (including some windows 
platforms) it can introduce delays of up to 200ms when talking over the 
wire (as opposed to localhost)
The second is the use of no buffering when reading the xmlrpc 
response.  It is done using the readline() call on a file-like object 
constructed from the socket.  When no buffering is in effect, this 
causes a separate recv() call for each character.

This patch fixes these issues separately:
1) We provide a new getheaderdata() function in httplib which just 
returns the data for the headers instead of sending it.  This can be 
used instead of endheaders() if the intention is to subsequently send 
more data.  xmlrpclib then uses this method of framing its POST 
messages.
2) We provide the named artgument bufsize=0 to the HTTPResponse class 
in httplib, and also so on for the HTTPConnection.getresponse() and 
HTTP.getreply(). xmlrpclib then passes bufsize=-1 to HTTP.getreply() to 
get default buffering for the response fileobject.

This patch focuses on fixing the problems with xmlrpclib.  but issue 1) 
above also has a number of other manifestations in the lib, there are 
other places where we can use getheaderdata() and send() instead of 
endheaders() to avoid possible Nagle/Ack problems, e.g. in urllib.py, 
distutils.py and loggin/handlers.py

----------
components: Extension Modules
files: xmlrpc.patch
keywords: easy, patch, patch
messages: 75962
nosy: krisvale
severity: normal
status: open
title: Fix performance issues in xmlrpclib
type: performance
versions: Python 2.7
Added file: http://bugs.python.org/file12030/xmlrpc.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue4336>
_______________________________________