[Python-ideas] TCP Fast Open protocol

[Terry Reedy]

On 1/10/2013 4:29 PM, Benoit Chesneau wrote:
>
> On Jan 10, 2013, at 10:24 PM, Guido van Rossum
> <guido at python.org
> <mailto:guido at python.org>> wrote:
>
>> Is there sample code for an HTTP client? What if the server doesn't
>> yet support the feature?
>
> Like I read it, this is transparent for the application if it doesn't
> support it.
>
> https://lwn.net/Articles/508865/

I read both the post (Aug 1, 2012, before the Linux 3.7 with the server 
code) and comments. FastOpen appears to still be an experimental 
proposal: "Currently, TFO is an Internet Draft with the IETF. ... (The 
current implementation employs the TCP Experimental Option Number 
facility as a placeholder for a real TCP Option Number.)". From the 
comments, I would say that its success outside of Google is not certain.

It appears that its main use case is repeated requests to webservers 
from browswers. This is because the latter often make *multiple* 
requests, often short, to the same site in order to construct a 
displayed web page. There is no time saving on the first request of a 
series. I suspect that after Google updates Chrome to use the new 
feature, one of the other 'independent' browsers is likely to be the 
next user.

To be active, the feature must be compiled into the socket code of both 
server and client machines AND must be explicitly requested by both 
client and server applications.

On the server side, it must be requested because the request makes a 
promise that syn+data requests will be handled idempotently. (So the 
default should be 'off'.) This is trivial for static web pages but may 
require app-specific overhead for anything else. So, in general, the app 
should not bother being able to handle FastOpen unless it will be run on 
servers with FastOpen, and for efficiency, it should not add the 
overhead unless it is needed because a particular request is from a 
FastOpen client.

This is not a problem for Google, with thousands of duplicate apps 
running on duplicate server configurations. But it was not clear in the 
OPs post how a Python app would know for sure whether a particular 
machine is FastOpen capable. I did not see the question of how a server 
app would know about the client connection type even addressed.

On the client side, .connect and at least the first .send must be 
combined into either .sendto or .sendmsg (which?, still to be decided, 
apparently;-) with a new MSG_FASTOPEN argument. So programs need a 
non-trivial rewrite. If a particular server is not fastopen capable, 
then new fastopen client kernal socket code can potentially handle the 
fallback to the old way. But if the client is not fastopen capable, the 
the fallback must be handled in the Python .sendto code or else in the 
client code. (So one of those layers must *know* the client system 
capability.)

Again, dealing with this, on multiple OSes, should be a lot easier for a 
monolithic browser like Chrome or Firefox (which might, on some systems, 
even use their own socket layer code), than for general purpose Python 
socket and app code.

So my conclusion is that this is (mostly) premature for Python at this 
time. This is a slight performance enhancement of limited use that will 
make code at least slightly more complex in a core module that must be 
keep at least as rock solid as it is now. Let Google get it working on 
both their servers and Chrome browser. And wait for Mozilla, say, to add 
it to Firefox. Things might change before the first 3.4 beta, but I 
think 3.5 is more likely. Of course, testing will require all 4 
combinations of client and server.

-- 
Terry Jan Reedy