Interprocess communication

[Murali]

Hi Python Gurus,

I am writing a GUI app (on linux) using pygtk which would launch some
external applications and display their stdout and stderr inside the
output window of my application synchronously. I am using the
subprocess module's Popen to launch the external programs and to
capture their stdout and stderr. The problem is that, for some
external programs that I launch inside my interface, I am not able to
capture and display the stdout as the program *runs*.

After some investigation, I found out that this problem had nothing to
do with my GUI app not getting refreshed and I was able to reproduce
this problem with normal python scripts. Here is one such script

#File test.py
from subprocess import Popen
from subprocess import PIPE
import sys
if __name__ == '__main__':
  prog = sys.argv[1]
  proc = Popen(prog, shell = True, stdout = PIPE, bufsize = 0)
  out = proc.stdout
  while proc.poll() is None:
    print out.readline()

Run this program as follows
$ test.py "ping -c 10 www.google.com"

Now, you would see the responses as if you just launched ping on the
command line. Now, lets look at another program. Here is a simple C++
program that prints numbers 1 to 10 at the passage of every second
(sort of a stopwatch)

#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>
main ( )
{
  timeval t1, t2;
  gettimeofday(&t1, NULL);
  int prev = -1;
  int cur = 0;
  while (true)
  {
    gettimeofday(&t2,NULL);
    if(t2.tv_sec - t1.tv_sec > 10)
      break;
    else
    {
      cur = t2.tv_sec-t1.tv_sec;
      if (cur != prev)
      {
        printf("%d\r\n",cur);
        prev = cur;
      }
    }
  }
}

if you would build this program and call it lets say timer ($ g++ -o
timer timer.cpp)  and run it with our python script like this

$python test.py "./timer"

you would see that every time you run the program your results vary
and on top of this the stdout of the timer program gets displayed all
at once presumably when the timer program has completed execution.

Why this discrepancy between the ping and timer programs? Is my
test.py script correct? Is there a better or a preferred method for
doing interprocess communication in Python.

Thanks!
Murali.