readline() blocks after select() says there's data??

[wealthychef]

"Donn Cave" <donn at u.washington.edu> wrote in message news:<a6tic4$uke$1 at nntp6.u.washington.edu>...
> Quoth wealthychef at mac.com (wealthychef):
> ...
> |> The complementary problem, of course, is that once you have invoked
> |> readline(), it will read all the data available on the file descriptor,
> |> or a lot of it anyway, and return only the first line.  The next time,
> |> it will return the next line from its buffer, but you have no way to
> |> know that it has another line waiting there.  select() can't see it.
> |> There are solutions, I think the easiest thing in most cases is to
> |> throw away the file object and just use the file descriptor (fileno())
> |> (or sockets in this case.)
> |
> | This is even more frightening.  I don't want to lose any data!  How
> | can I protect myself from this?  Why is using the file descriptor
> | better?
> 
> Well, you're not exactly losing data, it's just buffered, though that
> may be too fine a point depending on the circumstances.

Here's what I mean.  If I do a select.select() and it says, "there's
data," then I do a readline() on the file object, but select() was
talking about 2 lines of data, I will lose a line of data, because my
next select() will say "nothing left."  Is that right?  Because the
call to readline() somehow scarfs up all the data available and saves
it for my next call to readline()?  If that's so, then readline()
sucks, because I should have a way to query it to avoid blocking. 
Obviously select() and readline() just don't play ball together very
well...

> 
> The point with file descriptor is to use system I/O functions on the
> device, and avoid buffered C I/O.  Basically because select is a
> system I/O function.  If there were a C analogue to select, then you
> could use it with C buffered file objects, but there is no such thing.
> 
> When select tells you "this thing is ready to read", it means the
> device is ready for a system level read(), as in os.read(fd, bufsize).
> So do that, and you'll get what select was telling you about.  It's
> really simple.  The rules are the same, if you get an empty string
> it's at "end of file" (the pipe closed.)

I just want to be sure I understand this.  You're saying that
os.read() somehow get what select.select() was talking about, but that
f.fromchild.readline() will read less than that?

> 
> I have lost track of what kind of devices we're actually talking about -
> I'm seeing the word "socket", but then what looks like popen2.Popen3.
> Note that in Python, sockets are are normally socket objects, with recv()
> methods etc., but these are unbuffered and recv() is like os.read().
> Pipes (as created by Popen3) are either integer file descriptors or
> file objects, there isn't any special system level pipe object.
> As long as you're on UNIX, there's a certain purity of abstraction here
> that you can exploit if you want it just to simplify matters - you can
> get the file descriptor with s.fileno(), and you can use os.read() with
> that - or you can use sock.recv() if you prefer.

The objects from my original post are popen2.Popen3 objects, as you
noticed.  The Python lib manual says that os.read() should not be used
with file objects, even though I can get a file descriptor from them
with fileno().  Will it work?