[Web-SIG] Implementing File Upload Size Limits

[Robert Brewer]

Brian Smith wrote:
> Randy Syring wrote:
> > Hopefully you can clarify something for me.  Lets assume that the
> > client does not use '100 Continue' but sends data immediately, after
> > sending the headers.  If the server never reads the request content,
> > what does that mean exactly?  Does the data get transferred over the
> > wire but then discarded or does the client not get to send the data
> > until the server reads the request body?  I.e. the client tries to
> > "send" it, but the content isn't actually transferred across the
> > wire until the server reads it.  I am just wondering if there
> > is a buffer or queue or something between the server and the client
> > that allows data to be transferred even if the server doesn't
> > "read" the request body.  Or, is it just like a straight pipe
> > where one end (the client) can't push data through until the other
> > end (the server) reads it.
> 
> Under Apache CGI or mod_wsgi, in many situations you will get a
> deadlock in
> this scenario. The input and the output are buffered separately both
of
> those buffers can fill up. Neither mod_wsgi nor mod_cgid implement the
> non-blocking I/O logic needed to prevent deadlocks. I heard (but did
> not
> verify) that mod_fastcgi does not have this deadlocking problem. The
> sizes
> of the buffers determines the size of the inputs and outputs needed to
> cause
> a deadlock. On some platforms (e.g. Mac OS X), they are only 8K by
> default.
> 
> Therefore, for maximum portability, a WSGI application should ALWAYS
> consume
> the *whole* request body if it wants to avoid the deadlock using the
> reference WSGI adapter in PEP 333 or mod_wsgi.

Indeed. This is covered in RFC 2616 Section 8.2.3:

    If an origin server receives a request that does not include an
    Expect request-header field with the "100-continue" expectation,
    the request includes a request body, and the server responds
    with a final status code before reading the entire request body
    from the transport connection, then the server SHOULD NOT close
    the transport connection until it has read the entire request,
    or until the client closes the connection. Otherwise, the client
    might not reliably receive the response message. However, this
    requirement is not be construed as preventing a server from
    defending itself against denial-of-service attacks, or from
    badly broken client implementations.

CherryPy's wsgiserver will read any remaining request body (which the
application hasn't read) before sending response headers.


Robert Brewer
fumanchu at aminus.org