IPC

[Donn Cave]

In article <mailman.856.1090955408.5135.python-list at python.org>,
 <brianc at temple.edu> wrote:
> There's two modules that I like to use for this kind of stuff:
> http://starship.python.net/crew/tmick/ (process.py)
> http://www.lysator.liu.se/~astrand/popen5/ (also process.py
> just to be confusing)

And worse yet, neither does much processing per se.

> I recently did a quick hack using select and popen2, but I
> doubt it's fast or even safe, but it works.
> 
> #yadda yadda
> r, w, e = popen2.popen3(cmd)
> cmd = "?\n"
> w.write(cmd)
> w.flush()
> #continuely test for output to read
> while select.select([e],[],[],1)[0]:#timeout guessing game
>     line=e.readline()
>     #do something with line
> while select.select([r],[],[],1)[0]:#timeout guessing game
>     line=r.readline()
>     #do something with line
> 
> If you're output is line buffered you probably shouldn't have
> many problems with this approach if you can be assured your
> program will respond within the 1 second alloted to it. If
> you're positive you'll get output, just remove the timeout.

Two points spring to mind:

1.  Select's fundamental purpose, the reason you must
    write select([file], ... instead of select(file, ...
    is to handle multiple sources of input.

2.  The file object's fundamental purpose is to make
    sure select doesn't work.  Even if output is line
    buffered, that doesn't help input, which will still
    confound select with process buffering.  The buffer
    parameters for the input file objects could do
    something, but at the cost of reading input byte
    by byte at considerable expense in system calls.
    Use file descriptors.

Something like this would make more sense -

 efd = e.fileno()
 rfd = r.fileno()
 files = [rfd, efd]
 while files:
     rx, wx, ex = select(files, [], files, timeout)
     for f in rx:
         data = os.read(f, 4096)
         if data:
             if f == rfd:
                 rdata = rdata + data
             else:
                 edata = edata + data
         else:
             files.remove(f)
     for f in ex:
         print >> sys.stderr, 'error on unit', f
         files.remove(f)
 if edata:
     raise CommandError, edata
 do something with rdata.split('\n')

   Donn Cave, donn at u.washington.edu