Dialog with a process via subprocess.Popen blocks forever

[Gabriel Genellina]

En Fri, 02 Mar 2007 14:38:59 -0300, Donn Cave <donn at u.washington.edu>  
escribió:

> In article <mailman.4595.1172792873.32031.python-list at python.org>,
>  "Gabriel Genellina" <gagsl-py2 at yahoo.com.ar> wrote:
>
>> On http://docs.python.org/lib/popen2-flow-control.html there are some
>> notes on possible flow control problems you may encounter.
>
> It's a nice summary of one problem, a deadlock due to full pipe
> buffer when reading from two pipes.  The proposed simple solution
> depends too much on the cooperation of the child process to be
> very interesting, though.  The good news is that there is a real
> solution and it isn't terribly complex, you just have to use select()
> and UNIX file descriptor I/O.  The bad news is that while this is
> a real problem, it isn't the one commonly encountered by first
> time users of popen.

More bad news: you can't use select() with file handles on Windows.

>> If you have no control over the child process, it may be safer to use a
>> different thread for reading its output.
>
> Right - `I used threads to solve my problem, and now I have two
> problems.'  It can work for some variations on this problem, but
> not the majority of them.

Any pointers on what kind of problems may happen, and usual solutions for  
them?
On Windows one could use asynchronous I/O, or I/O completion ports, but  
neither of these are available directly from Python. So using a separate  
thread for reading may be the only solution, and I can't see why is it so  
bad. (Apart from buffering on the child process, which you can't control  
anyway).

-- 
Gabriel Genellina