non-blocking PIPE read on Windows

[Paul Du Bois]

placid wrote:
> What i need to do is, create a process using subprocess.Popen, where
> the subprocess outputs information on one line (but the info
> continuesly changes and its always on the same line) and read this
> information without blocking, so i can retrieve other data from the
> line i read in then put this in a GUI interface.

> readline() blocks until the newline character is read, but when i use
> read(X) where X is a number of bytes then it doesnt block(expected
> functionality) but i dont know how many bytes the line will be and its
> not constant so i cant use this too.

I wrote something for this the other day. The class has a getline()
method, which either returns a line or raises an exception instead of
blocking. It also raises an exception instead of returning EOF.

My use case had me reading from multiple processes at once; since
select() doesn't work on files in win32, I had to get a little cheesy.
I've appended the function that implements that use case, for
reference.

The central idea is to use PeekNamedPipe to figure out what's in the
pipe. You can then read that data without fear of blocking. I wrote it
quickly, therefore the code is a little embarassing, but... hopefully
useful all the same.

. class NoLineError(Exception): pass
. class NoMoreLineError(Exception): pass
. class liner(object):
.     """Helper class for multi_readlines."""
.     def __init__(self, f):
. 	  self.fd = f.fileno()
. 	  self.osf = msvcrt.get_osfhandle(self.fd)
. 	  self.buf = ''
.
.     def getline(self):
. 	  """Returns a line of text, or raises NoLineError, or
NoMoreLineError."""
. 	  try:
. 	      data, avail, _ = win32pipe.PeekNamedPipe(self.osf, 0)
. 	  except pywintypes.error:
. 	      # Pipe closed: give up what we have, then that's it
. 	      if self.buf:
. 		  ret, self.buf = self.buf, None
. 		  return ret
. 	      else:
. 		  raise NoMoreLineError
. 	  if avail:
. 	      self.buf += os.read(self.fd, avail)
.
. 	  idx = self.buf.find('\n')
. 	  if idx >= 0:
. 	      ret, self.buf = self.buf[:idx+1], self.buf[idx+1:]
. 	      return ret
. 	  else:
. 	      raise NoLineError
.
.
. def multi_readlines(fs):
.     """Read lines from |fs|, a list of file objects.
.     The lines come out in arbitrary order, depending on which files
.     have output available first."""
.     if type(fs) not in (list, tuple):
. 	  raise Exception("argument must be a list.")
.     objs = [liner(f) for f in fs]
.     for i,obj in enumerate(objs): obj._index = i
.     while objs:
. 	  for i,obj in enumerate(objs):
. 	      try:
. 		  yield (obj._index, obj.getline())
. 	      except NoLineError:
. 		  pass
. 	      except NoMoreLineError:
. 		  del objs[i]
. 		  break   # Because we mutated the array