Dialog with a process via subprocess.Popen blocks forever

[Donn Cave]

In article <mailman.4595.1172792873.32031.python-list at python.org>,
 "Gabriel Genellina" <gagsl-py2 at yahoo.com.ar> wrote:

> En Thu, 01 Mar 2007 14:42:00 -0300, <bayer.justin at googlemail.com> escribió:
> 
> > BUT If I use PIPE for both (so I can .write() on the stdin and .read()
> > from the subprocess' stdout stream (better: file descriptor)) reading
> > from the subprocess stdout blocks forever. If I write something onto
> > the subprocess' stdin that causes it to somehow proceed, I can read
> > from its stdout.
> 
> On http://docs.python.org/lib/popen2-flow-control.html there are some  
> notes on possible flow control problems you may encounter.

It's a nice summary of one problem, a deadlock due to full pipe
buffer when reading from two pipes.  The proposed simple solution
depends too much on the cooperation of the child process to be
very interesting, though.  The good news is that there is a real
solution and it isn't terribly complex, you just have to use select()
and UNIX file descriptor I/O.  The bad news is that while this is
a real problem, it isn't the one commonly encountered by first
time users of popen.

The more common problem, where you're trying to have a dialogue
over pipes with a program that wasn't written specifically to
support that, is not solvable per se - I mean, you have to use
another device (pty) or redesign the application.

> If you have no control over the child process, it may be safer to use a  
> different thread for reading its output.

Right - `I used threads to solve my problem, and now I have two
problems.'  It can work for some variations on this problem, but
not the majority of them.

   Donn Cave, donn at u.washington.edu