[Web-SIG] Server-side async API implementation sketches

Mon Jan 10 02:06:28 CET 2011

On 2011-01-09 09:03:38 -0800, P.J. Eby said:
> Hm.  I'm not sure if I like that.  The typical app developer really 
> shouldn't be yielding multiple body strings in the first place.

Wait; what?  So you want the app developer to load a 40MB talkcast MP3 
into memory before sending it?  You want to completely eliminate the 
ability to stream an HTML page to the client in chunks (e.g. <head> 
block, headers + search box, search results, advertisements, footer -- 
the exact thing Google does with every search result)?  That sounds 
like artificially restricting application developers, to me.

> I much prefer that the canonical example of a WSGI app just return a 
> list with a single bytestring...

Why is it wrapped in a list, then?

> IOW, I want it to look like the normal way to do thing is to just 
> return the whole request at once, and use the additional difficulty of 
> creating a second iterator to discourage people writing iterated bodies 
> when they should just write everything to a BytesIO and be done with it.

It sounds to me like your "should" doesn't cover an extremely large 
range of common use cases.

> In your approach, the above samples have to be rewritten as:
> 
>      return app(environ)
> 
> [snip]

My code does not use return.  At all.  Only yield.

> Try actually making some code that runs on this protocol and yields to 
> futures during the body iteration.

Sure.  I'll also implement my actual proposal of not having a separate 
body iterable.

> The above middleware pattern works with the sketches I gaveon the PEAK 
> wiki, and I've now updated the wiki to include an exampleapp and 
> middleware for clarity.

I'll need to re-read the code on your wiki; I find it incredibly 
difficult to grok, however, you can help me out a bit by answering a 
few questions about it: How does middleware trap exceptions raised by 
the application.  (Specifically how does the server pass the buck with 
exceptions?  And how does the exception get to the application to 
bubble out towards the server, through middleware, as it does now?)

> Really, the only hole in this approach is dealing with applications that block.

That's what the executor in the environ is for.  If you have image 
scaling or something else that will block you submit it.  All 
networking calls?  You submit them.

> The elephant in the room here is that while it's easy towrite these 
> example applications so they don't block, in practicepeople read files 
> and do database queries and what not in their requests, and those APIs 
> are generally synchronous.  So, unless they somehow fold their entire 
> application into a future, it doesn't work.

Actually, that's how multithreading support in marrow.server[.http] was 
implemented.  Overhead?  40-60 RSecs.  The option is provided for those 
who can do nothing about their application blocking, while still 
maintaining the internally async nature of the server.

> That you could never *call* the .read() method outside of a future,or 
> else you would block the server, thereby obliterating the point 
> ofhaving the async API in the first place.

See above re: your confusion over the calling semantics of wsgi.input 
in regards to my (and Alex's) proposal.  Specifically:

    data = (yield submit(wsgi_input.read, 4096)).result()

This would work on sync and async servers, and with sync and async 
applications, with no difference in the code.

	- Alice.