[Web-SIG] Re: Bill's comments on WSGI draft 1.4

[Phillip J. Eby]

At 11:41 AM 9/6/04 -0700, tony at lownds.com wrote:
>I'm specifically advocating that servers be required to use read() if they
>can't use fileno().

But with what block size?  If the block size is the whole file, why not 
just use:

     return [filelike.read()]

If it's some other block size, why not be explicit?


>  When an application returns an open file object,
>servers that send it out line by line (ie, as an interator) would be far
>far slower than servers that use fileno(). So that technique wouldn't
>really be portable across WSGI implementations. Using read() would make
>returning an open file a viable technique on all WSGI servers.

Okay, you've convinced me: the fileno() optimization (as it's currently 
specified) needs to be removed, and I need to strip out all mention of 
returning files from the application.  (Except maybe to mention that it's a 
bad idea!)

Instead of using 'fileno' as an extension attribute on the iterable, we'll 
add a 'wsgi.file_wrapper' key, usable as follows by an application:

     return environ['wsgi.file_wrapper'](something,blksize)

The 'file_wrapper' may introspect "something" in order to do a fileno() 
check, or other "I know how to send this kind of object quickly" 
optimizations.  It must return an iterable, that the application may return 
back to the server.  The server *must not* assume that the application 
*will* return the iterable; it is perfectly legal to do something like this:

     an_iter = environ['wsgi.file_wrapper'](something,blksize)

     for block in an_iter:
          yield block.replace('\n', '\r\n')

In this case, the application iterates over the file, but the original 
iterator's contents are not yielded.  In the same way, middleware may 
transform or ignore data yielded by the iterator.  So, in effect 
'file_wrapper' should just wrap the original file-like object in an 
iterator that the server can recognize and perform an optimization on, in 
the event that it *actually* is returned by the application.

Here's the simplest possible conforming implementation of 'file_wrapper', 
that works for any modern (1.5.2+) Python:

     class file_wrapper:

         def __init__(self,readable,blocksize=8192):
             self.readable, self.blocksize = readable, blocksize
             self.close = readable.close

         def __getitem__(self,index):
             data = self.readable.read(self.blocksize)
             if data:
                 return data
             raise IndexError

     environ['wsgi.file_wrapper'] = file_wrapper

     result = application(environ, start_response)

     if isinstance(result, file_wrapper):
         # check result.readable for fileno() or other optimizations
     else:
         # do normal iteration over 'result'

Unfortunately, this is a lot more boilerplate than I'd like to impose on 
server authors.  But, if we don't, then the same boilerplate is effectively 
imposed on all application/framework/middleware authors who want to return 
file-like objects.

The other hassle here is going to be adjusting the PEP's presentation 
sequence so that this complication doesn't obscure the simplicity of the 
"CGI Gateway" example.  :(

The other alternative is to check for a 'read()' method as an alternative 
to iterability, but it leaves open the question of appropriate block 
size.  I suppose we could say that this is up to the server.

But, no matter how the introspection works, it's going to work strongly 
against the appearance of simplicity in the examples.  :(