[Python-Dev] Making popen2 actually work

[Guido van Rossum]

> OK, so Guido told me at one point that 2.0 was supposed to have
> included changes to the way buffering of I/O to subprocesses is
> handled.  I've installed 2.0, and I need to know how to make the
> following program work:
> 
> import os, sys
> 
> (child_stdin, child_stdout) = os.popen2("/usr/bin/rev", "t", 1);
> print "* Child process spawned"
> 
> while 1:
>     line = raw_input(">");
>     child_stdin.write(line); child_stdin.flush()
>     print "* Line written to child process"
>     transformed = child_stdout.readline(); child_stdout.flush()
>     print "* Line read back from child process."
>     sys.stdout.write(transformed)
> 
> If anybody can tell me what the right magic is here, I'll write up 
> a patch for the library docs explaining it and including the corrected
> example.  This has to be a FAQ.

First, raw_input() strips the newline, so you should change the second
line of the loop body to

     child_stdin.write(line+'\n'); child_stdin.flush()

Second, with "cat" instead of "/usr/bin/rev", this works fine for me,
so your logic is fine.

The trick seems to be to get the subprocess not to buffer its output
(or input).  There's nothing that Python can do for you here!

Unfortunately, most programs use stdio for all their console I/O, and
stdio automatically buffers its output when connected to a pipe... :-(

If you absolutely, positively have to do this with a program that
won't unbuffer its output, your only resort would be to use pseudo
ttys -- but that's quite a different story... :-(

--Guido van Rossum (home page: http://www.python.org/~guido/)