Select weirdness

[Ron Garret]

In article <donn-58A627.10135523042007 at gnus01.u.washington.edu>,
 Donn Cave <donn at u.washington.edu> wrote:

> In article <rNOSPAMon-934744.11420922042007 at news.gha.chartermi.net>,
>  Ron Garret <rNOSPAMon at flownet.com> wrote:
> 
> > The answer is obvious: select is looking only at the underlying socket, 
> > and not at the rfile buffers.
> > 
> > So... is this a bug in select?  Or a bug in my code?
> 
> Yes.
> 
> I don't see any specific followup to this point, but it is or
> at least should be a well known limitation of C library I/O and
> select.  select() is an operating system function that has no
> way to know the state of your process I/O buffers, nor do the C
> stdio buffers don't come with a standard way to inspect that state.
> Therefore, if you mix C I/O with select(), you're more or less out
> of luck.  This applies directly to Python, because it calls the
> operating system select() function and it uses C stdio buffers for
> its file object (and entices you to make this mistake by supporting
> a file object as an input to select().)
> 
> This conflict can be relieved after a fashion by eliminating the
> buffer.  The now unbuffered C fgets / Python readline won't store
> extra lines that select can't see, but the semantics of the operation
> are still a poor fit with the usual application of select.  The
> number of system-level I/O calls is significantly greater as you
> transfer data one byte at a time, which may add a lot of overhead
> if you transfer a lot of data, and your readline() function still
> can't return until it gets that '\n', so a half line can block your
> application.
> 
> It isn't a lot of work to read data with operating system functions
> that are compatible with select - os.read(), socket.recv() - and
> break it up into lines on your own, and this completely and efficiently
> resolves the problem.

Yep, this is pretty much the conclusion I've come to.

As an editorial comment, I think it's quite confusing for Python's 
select to accept file objects without complaint if it's not going to do 
the Right Thing with them.

rg