[Web-SIG] PEP 444 / WSGI 2 Async

[Antoine Pitrou]

Alice Bevan–McGregor <alice at ...> writes:
> > Er, for the record, in Python 3 non-blocking file objects return None when
> > read() would block.
> 
> -1
> 
> I'm aware, however that's not practically useful.  How would you detect 
> from within the WSGI 2 application that the file object has become 
> readable?  Implement your own async reactor / select / epoll loop?  
> That's crazy talk!  ;)

I was just pointing out that if you need to choose a convention for signaling
blocking reads on a non-blocking object, it's already there.

By the way, an event loop is the canonical implementation of asynchronous
programming, so I'm not sure what you're complaining about. Or perhaps you're
using "async" in a different meaning? (which one?)

> >> agronholm: the requirements of async apps are a big problem
> >> 
> >> agronholm: returning magic values from the app sounds like a bad idea
> >> 
> >> agronholm: the best solution I can come up with is to have
> >> wsgi.async_input or something, which returns an async token for any
> >> given read operation
> > 
> > The idiomatic abstraction for non-blockingness under POSIX is file
descriptors.
> > So, at the low level (the WSGI level), exchanging fds between server and
app
> > could be enough to allow both to wake up each other (perhaps two fds: one
the
> > server can wait on, one the app can wait on). Similarly to what 
> > signalfd() does.
> > Then higher-level tools can wrap inside Futures or whatever else.
> 
> -0
> 
> Hmm; I'll have to mull that over.  Initial thoughts: having a magic 
> yield value that combines a fd and operation (read/write) is too 
> magical.

I don't understand why you want a "yield" at this level. IMHO, WSGI needn't
involve generators. A higher-level wrapper (framework, middleware, whatever) can
wrap fd-waiting in fancy generator stuff if so desired. Or, in some other
environments, delegate it to a reactor with callbacks and deferreds. Or whatever
else, such as futures.

By the way, the concurrent.futures module is new. Though it will be there in
3.2, it's not guaranteed that its API and semantics will be 100% stable while
people start to really flesh it out.

> +1 for pure futures which (in theory) eliminate the need for dedicated 
> async versions of absolutely everything at the possible cost of 
> slightly higher overhead.

I don't understand why futures would solve the need for a low-level async
facility. You still need to define a way for the server and the app to wake each
other (and for the server to wake multiple apps). This isn't done "naturally" in
Python (except perhaps with stackless or greenlets). Using fds give you
well-known flexible possibilities.

If you want to put the futures API in WSGI, think of the poor authors of a WSGI
server written in C who will have to write their own executor and future
implementation. I'm sure they have better things to do.

Regards

Antoine.