[Web-SIG] wsgiref questions

[Phillip J. Eby]

At 12:06 PM 12/20/2006 -0800, Guido van Rossum wrote:
>On 12/20/06, Phillip J. Eby <pje at telecommunity.com> wrote:
>>At 10:12 AM 12/20/2006 -0800, Guido van Rossum wrote:
>> >We're struggling to use wsgiref behind some Googlish infrastructure,
>> >and one of the issues we find is that there is code that forbids
>> >hop-by-hop headers. Unfortunately we have a reason to send one
>> >(Transfer-encoding).
>>
>>Are you sure it's not a Content-Encoding?
>
>No, "chunked" is definitely a transfer encoding.
>
>> >  What's the reason behind their prohibition, and
>> >what would be the right way to do this?
>>
>>See the thread here for the original rationale:
>>
>>http://mail.python.org/pipermail/web-sig/2004-September/000879.html
>
>Well, we're using wsgiref's simple_server which only speaks HTTP/1.0,
>but we really need to speak HTTP/1.1 and use chunked. Where do you
>recommend we put the chunking instead?

It's expected that an HTTP/1.1 server would handle chunking in the case 
where a WSGI app yields individual blocks, assuming that the client 
supports chunking and the server chooses to do it that way.  The way the 
spec is written, the server *must* ensure that each yielded chunk is 
transmitted to the client before the application is asked for the next 
chunk.  However, it's the server's choice as to whether the chunks will 
just be streamed or transmitted using chunked encoding.

wsgiref does have this comment at one point in the code (wsgiref.handlers), 
btw:

     # XXX Try for chunked encoding if origin server and client is 1.1

This is the place where you could try to add support for doing this.  I.e., 
that's the place where you'd add the Transfer-Encoding header, after 
checking for client support.

By the way, if your goal is just to ensure that data is written to the 
client as its yielded by the application, that can be done without needing 
the chunked encoding.  My understanding is that all chunked encoding is 
good for is sending data of unknown length over a persistent 
connection.  simple_server doesn't support persistent connections (and WSGI 
doesn't have "connections"), so chunking is of use only if you have some 
crazy kind of client that needs the chunking for some reason known only to 
its developers.  :)