[Web-SIG] PEP 444 (aka Web3)

[Armin Ronacher]

Hi,

On 2010-09-16 2:05 AM, P.J. Eby wrote:
> The first thing I notice is that web3.async appears to force all
> existing middleware to delete it from the environment if it wishes to
> remain compatible, unless it adapts to support receiving callables itself.
In terms of backwards compatibility, we have a huge change here anyways, 
so existing middlewares are not that much of an issue I support.  I know 
however that web3.async will be a controversial topic.

The reason it's in there is that there is theoretical support for async 
frameworks on top of WSGI to have some kind of basic interoperability.

Someone brought up the argument that it relies on polling, but that is 
only partially true because you control the incoming web3 environment. 
That environment might contain some callbacks that the application can 
use that would internally send signals to the server so that it knows 
when to call the response callable.

The callback was modeled after the hack that nginx (if I remember 
correctly) is doing wrt yielding empty strings until responses are ready.

I would like bring some people from asynchronous servers onto the 
discussion for that particular issue before we decide on the future. 
Tornado is currently the most popular Python project on github, so there 
is genuine interest in async servers and I am pretty sure enough people 
use it in practice.  This however also means that Tornado has its own 
environment which looks very much like the situation we were in before 
WSGI was around.

> On further reading I see you have something about middleware disabling
> itself if it doesn't support async execution, but this doesn't make any
> sense to me: if it can't support async execution, why wouldn't it just
> delete web3.async from the environ, forcing its wrapped app to be
> synchronous instead?
Instead of deleting it would set it to False though.  Why would it want 
to disable itself?  For instance because the middleware is actually 
depending on an asynchronous specification developed on top of web3 that 
is not supported by synchronous servers which is the main intention of 
that async flag.  To be used as the basis for an actual proper async 
specification written by people that actually use async servers unlike 
me and Chris :)

> My main fear with this spec is that people will assume they can just
> make a few superficial changes to run WSGI code on it, when in fact it
> is deeply incompatible where middleware is concerned. In fact, AFAICT,
> it seems like it will be *harder* to write correct web3 middleware than
> it is to write correct WSGI middleware now.
For just rewriting the environment it's about as complicated, and for 
making middlewares harder that modify the response I think this is a 
good thing.  Things middleware should do currently and do not:

- honour content-encoding
- correct set/unset content-length
- update/remove etags
- not be surprised by HEAD responses
- patching through exc_info
- not swallowing the write callable

I am sure there are more, I remember that Graham had some bad 
experiences with them in particular.

> This seems like a step backward, since the whole idea behind dropping
> start_response() was to make correct middleware *easier* to write.
Do we really need middlewares that rewrite the response?  Even without 
web3.async and limited to bytes only, there are so many things that can 
go wrong and will go wrong.  I would instead suggest a common library 
that people could use to develop middlewares on top of web3 that sorts 
these things out for you.


Regards,
Armin