[Web-SIG] WSGI: allowing short reads

[Antoine Pitrou]

Hi,

Robert Collins <robertc at ...> writes:
> 
> https://github.com/python-web-sig/wsgi-ng/issues/5
> 
> tl;dr - we don't specify whether read(size) has to return size bytes
> or just not more than size, today. the IO library is clear that
> read(n) returns up to n, and also offers read1 that guarantees only
> one read call.

I think you've got things mixed up. There are two different things in
"the IO library" (which is really the 3.x IO stack):

* buffered binary I/O has read(n) and read1(n):
  - read(n) will block until n bytes are received (except for non-blocking
    fds)
  - read1(n) will issue at most one system call and can return fewer than
    n bytes

* raw binary I/O only has read(n):
  - read(n) will issue at most one system call and can return fewer than
    n bytes

So, depending on which layer you are placing yourself on, one or the
other of your statements is wrong.

That said, it would be reasonable for WSGI to expose the raw I/O layer, 
IMHO. "Prettifying" libraries can wrap it inside a BufferedReader if
they like.

Note that I once proposed generalized prefetching on I/O streams, but
it was rejected:
https://mail.python.org/pipermail/python-ideas/2010-September/008179.html
(skip to the prefetch() proposal)

It was in the context of improving streamed unpickling, which is
a problem a bit similar - but less horrible - to JSON unserializing;
since then, the problem was solved in a different way by adding a
framing layer to pickle protocol 4 :-).

Regards

Antoine.