[Web-SIG] Bill's comments on WSGI draft 1.4

[Phillip J. Eby]

At 01:36 PM 9/2/04 -0700, Robert Brewer wrote:
>Phillip J. Eby wrote:
> > > I'd like to at least hear the rationale behind
> > > favoring iterables so heavily over write().
> >
> > One important reason: the server can suspend an iterable's execution
> > without tying up a thread.  It can therefore potentially use
> > a much smaller thread pool to handle a given number of connections,
> > because the threads are only tied up while they're executing an
> > iterator 'next()' call.
> >
> > By contrast, 'write()' occurs *within* the application execution,
> > so the only way to suspend execution is to suspend the thread (e.g.
> > waiting for a lock).
>
>Hmm. I still don't get it--why would the server not simply "suspend
>execution" of the framework within the write() call? In my naive
>estimation, it would be the difference between:
>
>for chunk in framework.data:
>     output(chunk)
>     do_out_of_band_stuff()
>
>..and:
>
>def write(chunk):
>     output(chunk)
>     do_out_of_band_stuff()

Because now you've moved the server code into the application thread; many 
Python web servers (pretty much all of the async ones including Medusa, 
Twisted, and ZServer) have a single thread for all I/O operations, distinct 
from the threads that run application requests.

So, if you want to perform I/O from an app thread, you need lock 
synchronization code that didn't exist before...  and the design rapidly 
becomes more complicated.

Anyway, such servers' write() methods will probably look more like:

    def write(self,data):
        self.output_queue.put(data)

and they'll then return to the caller.  However, this has new issues of its 
own: specifically, if a program transmits a large file, it will consume 
lots of memory if it produces data faster than the client can accept 
it.  (Because the output queue will back up.)

Of course, one can throttle the output queue to some set maximum size, but 
then you end up right where I began this discussion: the application thread 
has to hang, tying up that thread's availability until the app's execution 
is complete, and thus reducing the concurrent request throughput of the server.

If, however, the application is structured as an iterable, these problems 
all go away.  Application threads are only tied up for computation, not 
waiting for I/O, output a client isn't going to receive is never produced, 
large memory buffers aren't needed, and so on.

So, on purely technical grounds, the iterable approach is immensely 
superior; it should be used wherever practical to do so.


>..and in fact, I see most existing servers having to do both when they
>grow WSGI interfaces, since both are allowed in the WSGI spec (even if
>one is deprecated).

Yes, servers will have to support both; but it should be understood that 
for many important servers (especially ones written in Python) that 
applications using 'write()' may have detrimental effects on the server's 
overall throughput, even if the application seems to run quite well on say, 
a local connection to an unloaded server.

So, that's why people should be discouraged from using 'write()' outside of 
necessity.


>Maybe you could add a line or two of pseudocode to
>help me understand...? (Assuming you're not fleeing for your life from
>hurricanes, that is ;)

Hurricane's past me now; I just got power and 'net back this morning.