[Web-SIG] WSGI and greenlets

[Manlio Perillo]

Christopher Stawarz ha scritto:
> On May 6, 2008, at 6:17 AM, Manlio Perillo wrote:
> 
>> I'm glad to know that there are some other people interested in 
>> asynchronous application, do you have seen my extensions to WSGI in my 
>> module for Nginx?
> 
> Yes, I have, and I had your module in mind as a potential provider of 
> the AWSGI interface.
> 
>> Note that in Nginx the request body is pre-read before the application 
>> is called (in fact wsgi.input is either a cStringIO or File object).
> 
> Although I didn't state it explicitly in my spec, my intention is for 
> the server to be able to implement awsgi.input in any way it likes, as 
> long as it provides a recv() method.  It's totally acceptable for the 
> request body to be pre-read.
> 

Ok.
But what I meant was that since Nginx pre-read the request body I have 
not tried to implement an interface for dealing with an asynchronous 
wsgi.input ;-).


Moreover I don't see any readons to have a revc method instead of read.

>> Unfortunately there is a *big* usability problem: the extension is 
>> based on a well specified feature of WSGI: the gateway can suspend the 
>> execution of the WSGI application when it yields.
>>
>> However if the asynchronous code is present in a "child" function, we 
>> have something like this:
>> ...
>> That is, all the functions in the "chain" have to yield, and is not 
>> very good.
> 
> Yes, you're right.  However, if you're willing/able to use Python 2.5, 
> you can use the new features of generators to implement a call stack 
> that lets you call child functions and receive return values and 
> exceptions from them.  I've implemented this in awsgiref.callstack.  
> Have a look at
> 
>   
> http://pseudogreen.org/bzr/awsgiref/examples/echo_request_with_callstack.py
> 
> for an example of how it works.
> 

I don't think this will solve the problem.
Moreover in your example you buffer the whole request body so that you 
have to yield only one time.

>> The solution is to use coroutines, and I'm planning to integrate 
>> greenlets (from the pylib project) into the WSGI module for Nginx.
> 
> Interesting, but it's not clear to me how/if this would work.  Can you 
> explain more or point me to some code?
> 

http://codespeak.net/py/dist/greenlet.html

def process_commands(*args):
     while True:
         line = ''
         while not line.endswith('\n'):
             line += read_next_char()
         if line == 'quit\n':
             print "are you sure?"
             if read_next_char() != 'y':
                 continue    # ignore the command
         process_command(line)


With greenlets the execution can be suspened by any of the functions 
called by the main greelet.

This has a lot of advantages.

You can implement wsgi.input.read(n) so that it will suspend the 
execution of the current greenlet until *all* the n bytes have been read.

You can also implement the write callable so that control is returned to 
the main greelet when the socket is ready to send more data.

And, of course, you can implement a poll like interface and a sleep like 
interface.


I think that it is a great advantage, moreover it is the only way to 
implement truly reusable components.

Note that there is an effort of integrating greenlets with Twisted:
http://radix.twistedmatrix.com/2008/03/corotwine-01.html


The "problem" is that once you add support to greenlets, you have no 
more WSGI.

The interface can be the same, and applications can work on it without 
problems, but the semantic is *completely* different.


Also note that with greenlets should be possible to "magically" 
transform blocking applications like Django to non blocking.



The main problem I see with greenlet is that is is not yet stable (there 
are some problems with the garbage collector) and that is is not part of 
CPython.

This means that it can be not acceptable to write a PEP for a WSGI like 
interface with coroutine support.


> 
> Thanks,
> Chris
> 


Regards  Manlio Perillo