[Web-SIG] PEP 444 / WSGI 2 Async

[Antoine Pitrou]

Alice Bevan–McGregor <alice at ...> writes:
> 
> agronholm: what new features does pep 444 propose to add to pep 3333? \ 
> async, filters, no buffering?
> 
> GothAlice: Async, filters, no server-level buffering, native string 
> usage, the definition of "byte string" as "the format returned by 
> socket read" (which, on Java, is unicode!), and the allowance for 
> returned data to be Latin1 Unicode.

Regardless of the rest, I think the latter would be a large step backwards.
Clear distinction between bytes and unicode is a *feature* of Python 3.
Unicode-ignorant programmers should use frameworks which do the encoding work
for them.

(by the way, why you are targeting both Python 2 and 3?)

> agronholm: I'm not very comfortable with the idea of wsgi.input in 
> async apps \ I'm just thinking what would happen when you do 
> environ['wsgi.input'].read()
> 
> GothAlice: One of two things: in a sync environment, it blocks until it 
> can read, in an async environment [combined with yield] it 
> pauses/shelves your application until the data is available.

Er, for the record, in Python 3 non-blocking file objects return None when
read() would block. For example:

>>> r, w = os.pipe()
>>> flags = fcntl.fcntl(r, fcntl.F_GETFL, 0); fcntl.fcntl(r, fcntl.F_SETFL,
flags | os.O_NONBLOCK)
0
>>> os.read(r, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 11] Resource temporarily unavailable
>>> f = open(r, "rb")
>>> f.read(1) is None
True

> agronholm: the requirements of async apps are a big problem
> 
> agronholm: returning magic values from the app sounds like a bad idea
> 
> agronholm: the best solution I can come up with is to have 
> wsgi.async_input or something, which returns an async token for any 
> given read operation

The idiomatic abstraction for non-blockingness under POSIX is file descriptors.
So, at the low level (the WSGI level), exchanging fds between server and app
could be enough to allow both to wake up each other (perhaps two fds: one the
server can wait on, one the app can wait on). Similarly to what signalfd() does.
Then higher-level tools can wrap inside Futures or whatever else.

However, this also means Windows compatibility becomes more complicated, unless
the fds are sockets.

Regards

Antoine.