[Web-SIG] [RFC] x-wsgiorg.suspend extension

[Graham Dumpleton]

On 13 April 2010 18:22, Manlio Perillo <manlio_perillo at libero.it> wrote:
> Graham Dumpleton ha scritto:
>> [...]
>>> Just yielding an empty string does not give the server some important
>>> informations.
>>>
>>> As an example, with x-wsgi.suspend application can specify a timeout,
>>> that tells the server that the application must be resumed before
>>> timeout milliseconds have elapsed.
>>>
>>> And x-wsgi.suspend returns a callable that, when called, tell the server
>>> to poll the app again.
>>
>> There are other ways of doing that, the callable doesn't need to be in
>> the WSGI environment. This is because since it is single threaded, the
>> WSGI server need only record in a global variable for that WSGI
>> application some state about the current request. The separate
>> function to note the suspension can then lookup that and does what it
>> needs to. In other words, you don't need the WSGI environment to
>> maintain  that relationship.
>>
>
> This seems completely broken, to me; do you have looked at txwsgi
> implementation?
>
> It is true that the WSGI server is single threaded, but there can be
> multiple concurrent requests processed in this thread.
>
> What happens if one request is being suspended and a new one is being
> processed?
> As far as I can tell, the new request will note the suspend flag set to
> True, and will be suspended as well.

No. I said 'record in a global variable for that WSGI application some
state about the ***current request***'.

The WSGI server when it switches to calling into an application for
the purposes of a concurrent request, switches the global variable to
reference the state about the other request. So, when accessing via
that global variable from within an application, it is always only
looking at its own state.

The WSGI server obviously will be iterrogating all those active
request states to know which is not in a suspended state and which can
be called into.

>> Having the timeout as argument is also questionable anyway. All you
>> really need to do is to tell the WSGI server that I don't want to be
>> called until I tell it otherwise. The WSGI application could itself
>> handle the timeout in other ways.
>
> But I can't see the reason why this can not be done by
> x-wsgiorg.suspend, since it is a very convenient interface.
>
>> Overall one could do all of this without having to do anything in the
>> WSGI environment. As PJE points out, it can be done by relying only on
>> the ability to yield an empty string. Everything else can be in the
>> application realm with the application normally being bound to a
>> specific WSGI server/event loop implementation, thus no portability.
>>
>
> From what I can tell, this is only possible by having a custom variable
> in the WSGI environ

You may not be able to see the alternatives, but they definitely exist.

> But since I wrote txwsgi for precisely this reason, it should not be
> hard to prove that your idea is actually possible to implement (and it
> does not make implementation more complex as it should be, think about
> an implementation written in C).

The notion I am describing is no more difficult in C except to the
extent that writing against C API for Python is more verbose that pure
Python. This is going to be the case whatever you are doing and is how
the C API is and nothing to do with the solution.

>> The problem of a middleware not passing through an empty string
>> doesn't even need to be an issue in as much as the application could
>> track when it requested to be suspended and if called into again
>> before the required criteria had been met, it could detect a
>> middleware that wasn't playing by the rules and at least raise an
>> error rather than potentially go into blocking state and tight loop.
>>
>
> Yes.
> This is something that can be done by an implementation.
> Currently txwsgi only checks for suspend flag when an empty string is
> yielded by application.
>
>> One could theoretically abstract out an interface for a generic event
>> system, but what you don't want is a general purpose one. You want one
>> which is specifically associated with the concept of a WSGI server.
>
> Why?
> This is not required at all.

And neither is adding your suspend function to the WSGI environment.
You obviously are just not able to grok the bigger picture.

Sure one can have an extension with a very narrow focus which sort of
helps with in an issue, but if it doesn't address the bigger issues
and just perpuates the mess, it is not a good extension.

You show me a async extension for WSGI where you can take the exact
same application code and run it on a completely different async based
WSGI hosting mechanism, then I well listen, but your current idea
fails because your application is still inextricably wedded to the
event loop of the specific underlying framework. You have no
abstraction there to allow portability and this suspend proposal is
merely tinkering at the edges, not solving the real problems and
polluting the WSGI environment when there is no reason to.

Graham