Reading all buffered bytes without blocking

[Paul Moore]

On Tuesday, 3 March 2015 19:29:19 UTC, Serhiy Storchaka  wrote:
> On 03.03.15 18:07, Paul Moore wrote:
> > Is it possible to say to a BufferedReader stream "give me all the bytes you have available in the buffer, or do one OS call and give me everything you get back"? The problem is that the "number of bytes" argument to read1() isn't optional, so I can't do available_bytes = fd.read1().
> 
> Just specify large size.

Thanks. Looking at the source, it appears that a large size will allocate a buffer that size for the data even if the amount actually read is small (thinking about it, of couse it has to, doh, because the syscall needs it).

Anyway, it's a pretty microscopic risk in practice, and when I looked at them, the incremental codecs (codecs.iterdecode) really aren't that hard to use, so I can do it that way if it matters enough.

For what it's worth, in case anyone wants to know, incremental decoding looks like this:

def get():
    while True:
        data = process.stdout.read(1000)
        if not data:
            break
        yield data
for data in codecs.iterdecode(get(), encoding):
    sys.stdout.write(data)
    sys.stdout.flush()

Thanks.
Paul