[Web-SIG] WSGI for HTTP/2.0 ?

[Robert Collins]

On 20 September 2014 18:31, Graham Dumpleton <graham.dumpleton at gmail.com> wrote:
>
> On 20/09/2014, at 3:49 PM, Roberto De Ioris <roberto at unbit.it> wrote:
>
>> I can help a bit (i am the uWSGI lead developer and a nginx and Cherokee
>> contributor, and i have already implemented a spdy3 server last year)
>>
>> I honestly think that WSGI by itself needs a complete rewrite/rethink to
>> be adapted to modern (ok someone could say 'fashioned') patterns (that are
>> somewhat more 'urgent' than HTTP/2), but i agree that starting thinking
>> about HTTP/2 could be a good thing.
>
> I agree.
>
> Overhauling WSGI has more relevance because an underlying web server updating itself to support HTTP 2.0 will in the main have little relevance at the application layer as the web server is more than likely to have an adapter layer which makes things look the same to existing modules/protocol adapters.

I don't particular care what we call it -
extending/overhauling/replacing. Lets call the new API WSGI2 as a
shorthand for 'WSGI extended or overhauled to support things that are
way overdue'.

I think the key thiings we need are:
 - support HTTP/2
 - still support HTTP/1.x - This is reasonable because many
environments will want a single server accepting both HTTP/1.x and 2
at once with graceful degradation
 - support websockets
 - be backportable to Python2.7+
 - be adoptable by uWSGI/mod_wsgi/weurkzeug/gunicorn/Twisted's
thread-pool adapter etc
 - be approximately as easy to write middleware and glue for as WSGI has been
 - have a sane migration path for servers, middleware and apps: a
big-bang approach isn't going to fly with the huge install base of
WSGI
 - be able to write both forward and backwards compatibilty shims: that is,
   a) be able to write an adapter that exposes WSGI2 on the top and
pure, compatible WSGI on the bottom
   b) be able to write an adapter that exposes WSGI on the top and a
restricted WSGI2 on the bottom. (Where graceful degradation means
behaving the same as if an HTTP/1.x connection was being handled).

> In other words, Apache adding support for HTTP 2.0 isn't going to result in some sort of wholesale change of the Apache module interface, it would stay the same say whether HTTP 2.0 is used, especially just as an alternate way of doing the same thing as HTTP 1.1. In that respect, since no HTTP 2.0 specific functionality is going to be made visible through exist interfaces, then Apache modules or adapters for FASTCGI/SCGI etc or even mod_wsgi are simply not going to change.

The Apache module interface is a lot wider than WSGI today, so I'm
glad to hear you believe it won't need changing! WSGI has no
equivalent for the CONNECTION filter, for instance (and arguably
shouldn't because thats very much a server responsibility) - but we
need something to hack websockets and push responses on.

> So, overhaul WSGI as the primary aim, but within that factor in things to allow for HTTP 2.0 functionality.

I consider HTTP/2 one of potentially several primary aims: I'm
delighted to collaborate on overhauling WSGI, but if we overhaul it
and don't *deliver* HTTP/2, then I think thats going to be bad - there
is every opportunity for us to miss a fine detail and end up having to
rev our spec as soon as implementors find issues.

> The problem with trying to overhaul WSGI is that if it is done in an open forum like the Web-SIG it will die of a thousand cuts, as past efforts to update it in even minor ways have suffered.

Well, thats certainly a challenge :). Whats the governance model here?
Is a PEP appropriate, and if so - that gives us a BFDL or BFDL
PEP-delegate to decide between bikeshed issues; and if its not a
bikeshed issue then resolving it is actually necessary.

> The only way that WSGI itself will ever see an overhaul is through the strong willed determination of a few people off list, out of sight, to allow it it to be fully fleshed out, with input coming from direct consultation with and review by other related parties who have a vested interested or significant experience in the area.
>
> I may be up for such an off list effort, but be warned I may want to run roughshod over it and exert quite a lot of influence over the process and outcome. :-)

I will happily discuss stuff with you off-list, but I'm not
particularly interested in having the primary effort be cabal style -
HTTP/2 has managed to go through a much harder rev with very strong
personalities and much the same sort of death possible as you're
concerned about here with great transparency.

I don' t think thats incompatible with your needs though - for
instance, if you want to stay offlist and debate privately to avoid
1000-cut-pain : thats fine, but I reserve the right to summarise and
discuss things here (and equally to ignore kibbitzing here that isn't
being productive).

As far as qualifications, I've a long history with HTTP (about 14
years now - I joined the Squid project circa 2000), applications that
will benefit from it (horizon, the OpenStack APIs and lmirror), varied
experience deploying WSGI and WSGI derived things, and writing code
both inside frameworks like Django and Zope3, as well as straight to
WSGI.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Converged Cloud