[Web-SIG] PEP 444 / WSGI 2 Async

[Alex Grönholm]

07.01.2011 07:24, Alex Grönholm kirjoitti:
> 07.01.2011 06:49, P.J. Eby kirjoitti:
>> At 05:47 PM 1/6/2011 -0800, Alice Bevan­McGregor wrote:
>>> Tossing the idea around all day long will then, of course, be 
>>> happening regardless.  Unfortunately for that particular discussion, 
>>> PEP 3148 / Futures seems to have won out in the broader scope.
>>
>> Do any established async frameworks or server (e.g. Twisted, 
>> Eventlet, Gevent,  Tornado, etc.) make use of futures?
> I understand that Twisted has incorporated futures support to their 
> deferreds. Others, I believe, don't support them yet. You have to 
> consider that Python 3.2 (the first Python with futures support in 
> stdlib) hasn't even been released yet, and it's only been two weeks 
> since I released the drop-in backport 
> (http://pypi.python.org/pypi/futures/2.1).
Exarkun corrected me on this -- there is currently no futures support in 
Twisted. Sorry about the false information.
>
>>
>>
>>>   Having a ratified and incorporated language PEP (core in 3.2 w/ 
>>> compatibility package for 2.5 or 2.6+ support) reduces the scope of 
>>> async discussion down to: "how do we integrate futures into WSGI 2" 
>>> instead of "how do we define an async API at all".
>>
>> It would be helpful if you addressed the issue of scope, i.e., what 
>> features are you proposing to offer to the application developer.
>>
>> While the idea of using futures presents some intriguing 
>> possibilities, it seems to me at first glance that all it will do is 
>> move the point where the work gets done.  That is, instead of simply 
>> running the app in a worker, the app will be farming out work to 
>> futures.  But if this is so, then why doesn't the server just farm 
>> the apps themselves out to workers?
>>
>> I guess what I'm saying is, I haven't heard use cases for this from 
>> the application developer POV -- why should an app developer care 
>> about having their app run asynchronously?
> Applications need to be asynchronous to work on a single threaded 
> server. There is no other benefit than speed and concurrency, and 
> having to program a web app to operate asynchronously can be a pain. 
> AFAIK there is no other way if you want to avoid the context switching 
> overhead and support a huge number of concurrent connections.
>
> Thread/process pools are only necessary in an asynchronous application 
> where the app needs to use blocking network APIs or do heavy 
> computation, and such uses can unfortunately present a bottleneck. It 
> follows that it's pretty pointless to have an asynchronous application 
> that uses a thread/process pool on every request.
>
> The goal here is to define a common API for these mutually 
> incompatible asynchronous servers to implement so that you could one 
> day run an asynchronous app on Twisted, Tornado, or whatever without 
> modifications.
>>
>> So far, I believe you're the second major proponent (i.e. ones with 
>> concrete proposals and/or implementations to discuss) of an async 
>> protocol...  and what you have in common with the other proponent is 
>> that you happen to have written an async server that would benefit 
>> from having apps operating asynchronously.  ;-)
>>
>> I find it hard to imagine an app developer wanting to do something 
>> asynchronously for which they would not want to use one of the 
>> big-dog asynchronous frameworks.  (Especially if their app involves 
>> database access, or other communications protocols.)
>>
>> This doesn't mean I think having a futures API is a bad thing, but 
>> ISTM that a futures extension to WSGI 1 could be defined right now 
>> using an x-wsgi-org extension in that case...  and you could then 
>> find out how many people are actually interested in using it.
>>
>> Mainly, though, what I see is people using the futures thing to 
>> shuffle off compute-intensive tasks...  but if they do that, then 
>> they're basically trying to make the server's life easier...  but 
>> under the existing spec, any truly async server implementing WSGI is 
>> going to run the *app* in a "future" of some sort already...
>>
>> Which means that the net result is that putting in async is like 
>> saying to the app developer: "hey, you know this thing that you just 
>> could do in WSGI 1 and the server would take care of it for you?  
>> Well, now you can manage that complexity by yourself!  Isn't that 
>> wonderful?"   ;-)
>>
>> I could be wrong of course, but I'd like to see what concrete use 
>> cases people have for async.  We dropped the first discussion of 
>> async six years ago because someone (I think it might've been James) 
>> pointed out that, well, it isn't actually that useful.  And every 
>> subsequent call for use cases since has been answered with, "well, 
>> the use case is that you want it to be async."
>>
>> Only, that's a *server* developer's use case, not an app developer's 
>> use case...  and only for a minority of server developers, at that.
>>
>> _______________________________________________
>> Web-SIG mailing list
>> Web-SIG at python.org
>> Web SIG: http://www.python.org/sigs/web-sig
>> Unsubscribe: 
>> http://mail.python.org/mailman/options/web-sig/alex.gronholm%40nextday.fi
>
> _______________________________________________
> Web-SIG mailing list
> Web-SIG at python.org
> Web SIG: http://www.python.org/sigs/web-sig
> Unsubscribe: 
> http://mail.python.org/mailman/options/web-sig/alex.gronholm%40nextday.fi