Running a command line program and reading the result as it runs

[Antoon Pardon]

Op 22-08-13 07:51, Ian Simcock schreef:
> Greetings all.
> 
> I'm using Python 2.7 under Windows and am trying to run a command line
> program and process the programs output as it is running. A number of
> web searches have indicated that the following code would work.
> 
> import subprocess
> 
> p = subprocess.Popen("D:\Python\Python27\Scripts\pip.exe list -o",
>                      stdout=subprocess.PIPE,
>                      stderr=subprocess.STDOUT,
>                      bufsize=1,
>                      universal_newlines=True,
>                      shell=False)
> for line in p.stdout:
>     print line
> 
> When I use this code I can see that the Popen works, any code between
> the Popen and the for will run straight away, but as soon as it gets to
> the for and tries to read p.stdout the code blocks until the command
> line program completes, then all of the lines are returned.
> 
> Does anyone know how to get the results of the program without it blocking?

Maybe the following can work?

Untested code:

from pty import openpty
from subprocess import Popen

master, slave = openpty()

p = Popen("D:\Python\Python27\Scripts\pip.exe list -o",
          stdout = slave,
          stderr = slave,
          stdin = slave,
          close_fds = True)

for line in master:
    print line


The idea is to set a a pseudo terminal for pip so that the
system thinks pip is doing IO with a terminal and so the
IO will be line buffered. But all IO from pip will be available
through the master in your program.

-- 
Antoon Pardon