read stdout/stderr without blocking

[Grant Edwards]

On 2005-09-13, Jacek Pop?awski <jpopl at interia.pl> wrote:
> Grant Edwards wrote:
>> You're right.  I must have been remembering the behavior of a
>> network socket.  Apparently, you're supposed to read a single
>> byte and then call select() again.  That seems pretty lame.
>
> I created another thread with single read(), it works, as long
> as I have only one PIPE (i.e. stderr is redirected into
> stdout). I wonder is it Python limitation or systems one (I
> need portable solution)?

Not sure what you mean.  Here is my test program that blocks on
the read(1024) call:

#!/usr/bin/python
import os,select

p = os.popen("while sleep 2; do date; done","r")
print p

while 1:
    r,w,e = select.select([p],[],[],1)
    if r:
        d = r[0].read(1024)
        print len(d),repr(d)
    else:
        print "timeout"

It also blocks if the call is changed to read().  This seems
pretty counter-intuitive, since that's not the way read()
usually works on pipes.
        
Here's the corresponding C program that works as I
expected (read(1024) returns available data):

#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <sys/select.h>

unsigned char buffer[1024];
int main(void)
{
  fd_set readfds, writefds, exceptfds;
  struct timeval tv;
  FILE *fp;
  int fd;
  
  fp = popen("while sleep 2; do date; done","r");
  if (!fp)
    {
      perror("popen");
      exit(1);
    }
  fd = fileno(fp);
  
  FD_ZERO(&readfds);
  FD_ZERO(&writefds);
  FD_ZERO(&exceptfds);
  
  while (1)
    {
      int s;
      FD_SET(fd,&readfds);
      tv.tv_sec = 1;
      tv.tv_usec = 0;
      s = select(fd+1,&readfds,&writefds,&exceptfds,&tv);
      if (s==0)
        printf("timeout\n");
      else if (s<0)
        {
          perror("select");
          exit(2);
        }
      else
        {
          if FD_ISSET(fd,&readfds)
            {
              int n = read(fd,buffer,(sizeof buffer)-1);
              buffer[n] = '\0';
              printf("read %d: '%s'\n",n,buffer);
            }
        }
    }
}


-- 
Grant Edwards                   grante             Yow!  Does that mean
                                  at               I'm not a well-adjusted
                               visi.com            person??