[Web-SIG] [RFC] x-wsgiorg.suspend extension

[Graham Dumpleton]

On 12 April 2010 21:25, Manlio Perillo <manlio_perillo at libero.it> wrote:
> Graham Dumpleton ha scritto:
>> On 12 April 2010 06:07, Manlio Perillo <manlio_perillo at libero.it> wrote:
>>> I'm not sure about the correct procedure to follow, I hope it is not a
>>> problem.
>>>
>>> I here propose the x-wsgiorg.suspend to be accepted as official WSGI
>>> extension, using the wsgiorg namespace.
>>>
>
> First of all thanks for the feedback.
>
>> [...]
>> In the code of demo_fdevent.py it has:
>>
>>     while True:
>>         while True:
>>             ret, num_handles = m.perform()
>>             if ret != pycurl.E_CALL_MULTI_PERFORM:
>>                 break
>>         if not num_handles:
>>             break
>>
>>         read, write, exc = m.fdset()
>>         resume = environ['x-wsgiorg.suspend'](1000)
>>         if read:
>>             readable(read[0], resume)
>>             yield ''
>>         else:
>>             writeable(write[0], resume)
>>             yield ''
>>
>> The registration of file descriptors doesn't occur until after the
>> first suspend() call.
>>
>> If the underlying reactor that the WSGI server is presumably also
>> using doesn't know about the file descriptors at that point, then how
>> does it now to return from the suspend().
>>
>
> I'm not sure to understand your concern, but the execution is not
> suspended when you call x-wsgiorg.suspend, but only when you yield a
> empty string.

Okay, missed that.

> In the example, registration of file descriptor occur before application
> is suspended.
>
>> You are also calling perform() before that point. When calling that,
>> it is presumed you have already done a select/poll to know data is
>> available, but you haven't done that on first pass through the loop.
>> If you call that and data isn't ready, can't it block still.
>>
>
> I have to admit that I just copied the example from fdevent specification.
> However the code seems correct, to me.
>
>> This example also illustrates well why I am so against an asynchronous
>> WSGI server extension.
>>
>> The reason is that your specific application has to be with this
>> extension bound to the specific event loop mechanism used by the
>> underlying WSGI server.
>>
>> I can't for example take this application and host it on a different
>> WSGI server which implements the same WSGI extension but uses a
>> different event loop.
>>
>
> Instead I think that being "agnostic" about how it is used, in one of
> the most important feature of x-wsgiorg.suspend extension.
>
> After all, if you think about it, how to interface with a database in a
> WSGI application is not specified by WSGI.
> This is done by a separate standard, dbapi2.
>
> For applications that need a template engine, we don't even have a
> standard inteface.
>
> The lack of a standard event API is not a problem that should be
> discussed in WSGI.
> It is a problem with the Python community; in fact I would like to
> define a standard event API *and* a standard efficient network API (the
> reason is expressed at the end of the README file in txwsgi).
>
>> If one can't do that and it is tied to the event loop and
>> infrastructure of the underlying WSGI server, what is the point of
>> defining and implementing the WSGI extension as it doesn't aid
>> portability at all, so what service is it actually providing?
>>
>
> The service it provides is: "allow a WSGI application to suspend its
> execution and resume it later".
>
>> In that respect, the extension:
>>
>> http://www.wsgi.org/wsgi/Specifications/fdevent/
>>
>> provided more as at least it tried to abstract out a generic interface
>> for registering interest in file descriptor activity and so perhaps
>> allow the application not to be dependent on the specific event loop
>> used by the underlying WSGI server.
>>
>
> However exposing this event interface is really something that has
> little to do with WSGI.
>
> Moreover, the fdevent example is rather inefficient.
> Suspensions should be minimized, and this is not possible with
> x-wsgiorg.fdevent but it is possible with x-wsgiorg.suspend.
>
>>>From the open issues of that other specification however, you can see
>> that there can be problems. It only allowed an application to be
>> interested in a single file descriptor where some packages may need to
>> express interest in more than one.
>>
>> Quite often an application is never going to be that simple anyway.
>> Some event systems allow a lot more than just watching of file
>> descriptors and timeouts however. You cant come up with a generic
>> interface for all these as they will not be able to be implemented by
>> a different event system which isn't so feature rich or which has a
>> different style of interface. Thus applications are restricted to the
>> lowest common denominator and likely that is not going to be enough
>> for most and so have no choice but to bind it to interfaces of
>> specific event loop.
>
> This is the reason why x-wsgiorg.resume is a better API than the one
> proposed by x-wsgiorg.fdevent, IMHO.
>
>> If that is going to be the case anyway, you may
>> as well forget about WSGI and write to that event systems specific web
>> server interface.
>>
>> So, given that one of the strengths of WSGI is that it is an interface
>> which aids portability of applications to different hosting
>> mechanisms, explain to me what purpose this WSGI extension has if it
>> doesn't aid portability given that your application still has to be
>> aware of the underlying event loop of that specific system anyway.
>>
>
> I suspect that it is not clear what the real purpose of
> x-wsgiorg.suspend extension is.
>
> The purpose of the extension if to just have a standard interface that
> WSGI applications can use to take advantage of the possibility, offered
> by asynchronous server, to suspend execution and resume it later.
>
> In the specification text I have explicitly stated that how resume is
> called is not specified.
>
>
> Claiming that x-wsgiorg.suspend does not help writing portable WSGI
> application is something similar (well, I'm a bit exaggerating here) of
> saying that WSGI does not allow to write portable web applications,
> because real world WSGI applications needs a database, a database
> engine, and so on.

It is not the same. I can take code using a specific database instance
and still run that WSGI application, using the same database, on a
different WSGI hosting mechanism without really changing anything
about how I interact with the WSGI server and its request handling.
The concern here is the WSGI interface and interacting with the web
server, not other non related third party packages.

You are articificially adding something to the WSGI interface as an
extension which is pointless. Since you are bound to the specific
event loop of the underlying WSGI server or event framework being used
you may just as well call a function directly on the WSGI server.
Adding that function under a key in the WSGI environment and accessing
it that way does not in itself provide any value and doesn't somehow
make the code easily portable to a different WSGI hosting mechanism
using a different event loop as you still have to change lots of other
code in your application.

In some respects this is similar to the issues between using a WSGI
wrapper which injects stuff in WSGI environment versus that
functionality being in a separate library. Read:

  http://dirtsimple.org/2007/02/wsgi-middleware-considered-harmful.html

Graham