[Web-SIG] Proposed WSGI extensions for asynchronous servers

[Christopher Stawarz]

This is a revised version of my AWSGI proposal from last week.  While  
many of the details remain the same, the big change is that I'm now  
proposing a set of extensions to standard WSGI, rather than a separate  
specification for asynchronous servers.

The updated proposal is included below.  I've also posted it at

   http://wsgi.org/wsgi/Specifications/async

The bzr repository for my reference implementation (which is only  
partially updated to match the new spec) is now at

   http://pseudogreen.org/bzr/wsgiorg_async_ref/

I'd appreciate your comments.


Thanks,
Chris



Abstract
--------

This specification defines a set of extensions that allow WSGI
applications to run effectively on asynchronous (aka event driven)
servers.

Rationale
---------

The architecture of an asynchronous server requires all I/O
operations, including both interprocess and network communication, to
be non-blocking.  For a WSGI-compliant server, this requirement
extends to all applications run on the server.  However, the WSGI
specification does not provide sufficient facilities for an
application to ensure that its I/O is non-blocking.  Specifically,
there are two issues:

* The methods provided by the input stream (``environ['wsgi.input']``)
   follow the semantics of the corresponding methods of the ``file``
   class.  In particular, each of these methods can invoke the
   underlying I/O function (in this case, ``recv`` on the socket
   connected to the client) more than once, without giving the
   application the opportunity to check whether each invocation will
   block.

* WSGI does not provide the application with a mechanism to test
   arbitrary file descriptors (such as those belonging to sockets or
   pipes opened by the application) for I/O readiness.

This specification defines a standard interface by which asynchronous
servers can provide the required facilities to applications.

Specification
-------------

Servers that want to allow applications to perform non-blocking I/O
must add four new variables to the WSGI environment:
``x-wsgiorg.async.input``, ``x-wsgiorg.async.readable``,
``x-wsgiorg.async.writable``, and ``x-wsgiorg.async.timeout``.  The
following sections describe these extensions.

Non-blocking Input Stream
~~~~~~~~~~~~~~~~~~~~~~~~~

The ``x-wsgiorg.async.input`` variable provides a non-blocking
replacement for ``wsgi.input``.  It is an object with one method,
``read(size)``, that behaves like the ``recv`` method of
``socket.socket``.  This means that a call to ``read`` will invoke the
underlying socket ``recv`` **no more than once** and return **at
most** ``size`` bytes of data (possibly less).  In addition, ``read``
may return an empty string (zero bytes) **only** if the client closes
the connection or the application attempts to read more data than is
specified by the ``CONTENT_LENGTH`` variable.

Before each call to ``read``, the application **must** test the input
stream for readiness with ``x-wsgiorg.async.readable`` (see below).
The result of calling ``read`` on a non-ready input stream is
undefined.

As with ``wsgi.input``, the server is free to implement
``x-wsgiorg.async.input`` using any technique it chooses (performing
reads on demand, pre-reading the request body, etc.).  The only
requirements are for ``read`` to obey the expected semantics and the
input object to be accepted as the first argument to
``x-wsgiorg.async.readable``.

Testing File Descriptors for I/O Readiness
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The variables ``x-wsgiorg.async.readable`` and
``x-wsgiorg.async.writable`` are callable objects that accept two
positional arguments, one required and one optional.  In the following
description, these arguments are given the names ``fd`` and
``timeout``, but they are not required to have these names, and the
application **must** invoke the callables using positional arguments.

The first argument, ``fd``, is either an integer representing a file
descriptor or an object with a ``fileno`` method that returns such an
integer.  (In addition, ``fd`` may be ``x-wsgiorg.async.input``, even
if it lacks a ``fileno`` method.)  The second, optional argument,
``timeout``, is either ``None`` or a floating-point value in seconds.
If omitted, it defaults to ``None``.

When called, ``readable`` and ``writable`` return the empty string
(``''``), which **must** be yielded by the application iterable to the
server (passing through any middleware).  The server then suspends
execution of the application until one of the following conditions is
met:

* The specified file descriptor is ready for reading or writing.

* ``timeout`` seconds have elapsed without the file descriptor
   becoming ready for I/O.

* The server detects an error or "exceptional" condition (such as
   out-of-band data) on the file descriptor.

Put another way, if the application calls ``readable`` and yields the
empty string, it will be suspended until
``select.select([fd],[],[fd],timeout)`` would return.  If the
application calls ``writable`` and yields the empty string, it will be
suspended until ``select.select([],[fd],[fd],timeout)`` would return.

If ``timeout`` seconds elapse without the file descriptor becoming
ready for I/O, the variable ``x-wsgiorg.async.timeout`` will be true
when the application resumes.  Otherwise, it will be false.  The value
of ``x-wsgiorg.async.timeout`` when the application is first started
or after it yields each response-body string is undefined.

The server may use any technique it desires to detect when an
application's file descriptors are ready for I/O.  (Most likely, it
will add them to the same event loop that it uses for accepting new
client connections, receiving requests, and sending responses.)

Examples
--------

The following application reads the request body and sends it back to
the client unmodified.  Each time it wants to receive data from the
client, it first tests ``environ['x-wsgiorg.async.input']`` for
readability and then calls its ``read`` method.  If the input stream
is not readable after one second, the application sends a ``408
Request Timeout`` response to the client and terminates::

   def echo_request_body(environ, start_response):
       input = environ['x-wsgiorg.async.input']
       readable = environ['x-wsgiorg.async.readable']

       nbytes = int(environ.get('CONTENT_LENGTH') or 0)
       output = ''
       while nbytes:
           yield readable(input, 1.0)  # Time out after 1 second

           if environ['x-wsgiorg.async.timeout']:
               msg = 'The request timed out.'
               start_response('408 Request Timeout',
                              [('Content-Type', 'text/plain'),
                               ('Content-Length', str(len(msg)))])
               yield msg
               return

           data = input.read(nbytes)
           if not data:
               break
           output += data
           nbytes -= len(data)

       content_type = (environ.get('CONTENT_TYPE') or 'application/ 
octet-stream')
       start_response('200 OK', [('Content-Type', content_type),
                                 ('Content-Length', str(len(output)))])
       yield output

The following middleware component allows an application that uses the
``x-wsgiorg.async`` extensions to run on a server that does not
support them, without any modification to the application's code::

   def dummy_async(application):
       def wrapper(environ, start_response):
           input = environ['wsgi.input']
           environ['x-wsgiorg.async.input'] = input

           select_args = [None]

           def readable(fd, timeout=None):
               select_args[0] = ([fd], [], [fd], timeout)
               return ''

           def writable(fd, timeout=None):
               select_args[0] = ([], [fd], [fd], timeout)
               return ''

           environ['x-wsgiorg.async.readable'] = readable
           environ['x-wsgiorg.async.writable'] = writable

           for result in application(environ, start_response):
               if result or (not select_args[0]):
                   yield result
               else:
                   if select_args[0][2][0] is input:
                       environ['x-wsgiorg.async.timeout'] = False
                   else:
                       ready = select.select(*select_args[0])
                       environ['x-wsgiorg.async.timeout'] = (ready ==  
([],[],[]))
                   select_args[0] = None

       return wrapper

Problems
--------

* The empty string yielded by an application after calling
   ``readable`` or ``writable`` must pass through any intervening
   middleware and be detected by the server.  Although WSGI explicitly
   requires middleware to relay such strings to the server (see
   `Middleware Handling of Block Boundaries
   <http://python.org/dev/peps/pep-0333/#middleware-handling-of-block-boundaries 
 >`_),
   some components may not, making them incompatible with this
   specification.

* Although the extensions described here make it *possible* for
   applications to run effectively on asynchronous servers, they do not
   (and cannot) *ensure* that they do so.  As is the case with any
   cooperative multitasking environment, the burden of ensuring that
   all application code is non-blocking rests with application authors.

Other Possibilities
-------------------

* To prevent an application that does blocking I/O from blocking the
   entire server, an asynchronous server could run each instance of the
   application in a separate thread.  However, since asynchronous
   servers achieve high levels of concurrency by expressly *avoiding*
   multithreading, this technique will almost always be unacceptable.

* The `greenlet <http://codespeak.net/py/dist/greenlet.html>`_ package
   enables the use of cooperatively-scheduled micro-threads in Python
   programs, and a WSGI server could potentially use it to pause and
   resume applications around blocking I/O operations.  However, such
   micro-threading is not part of the Python language or standard
   library, and some server authors may be unwilling or unable to make
   use of it.

Open Issues
-----------

* Some third-party libraries (such as `PycURL
   <http://pycurl.sourceforge.net/>`_) provide non-blocking interfaces
   that may need to monitor multiple file descriptors for I/O readiness
   simultaneously.  Since this specification allows an application to
   wait on only one file descriptor at a time, it may be difficult or
   impossible for applications to use such libraries.

   Although this specification could be extended to include an
   interface for waiting on multiple file descriptors, it is unclear
   whether it would be easy (or even possible) for all servers to
   implement it.  Also, the appropriate behavior for a multi-descriptor
   wait is not obvious.  (Should the application be resumed when a
   single descriptor is ready?  All of them?  Some minimum number?)