Running a command line program and reading the result as it runs

[Grant Edwards]

On 2013-08-22, Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, Aug 23, 2013 at 1:26 AM, Ian Simcock
><Ian.Simcock at internode.on.net> wrote:
>> Chris Angelico wrote:
>>>
>>> Is the program actually producing output progressively? I just tried
>>> your exact code with "dir /ad /s /b" and it worked fine, producing
>>> output while the dir was still spinning (obviously setting shell=True
>>> to make that work, but I don't think that'll make a difference). It
>>> may be that pip buffers its output. Is there a parameter to pip to
>>> make it pipe-compatible?
>>>
>>> ChrisA
>>>
>>
>> If I run pip in the command window I can see it's output appearing line by
>> line rather than on one block.
>>
>> I tried the code with the dir command but it's too fast for me to be sure if
>> it's working or not.
>>
>> I tried again using the command "ping google.com" instead since I know that
>> output's slowly and it something that everyone should have. In the command
>> window I can see that the output appears over time, but from python I get
>> nothing for a while and then suddenly get all the output in one rapid go.
>>
>>
>> Can you think of anything else I can look at?
>
> A lot of programs, when their output is not going to the console,
> will buffer output. It's more efficient for many purposes. With Unix
> utilities, there's often a parameter like --pipe or --unbuffered that
> says "please produce output line by line", but Windows ping doesn't
> have that - and so I'm seeing the same thing you are.

Another way this problem can be avoided on Unix is to connect the
slave end of a pty (instead of a pipe) to the command's stdout/stderr
and then read the command's output from the master end of the pty.
[On Unix, the buffering decision is based on whether stdout is a tty
device, not on whether it's the console.]

Dunno whether Windows has ptys or not.  They're a very simple, elegent
solution to a number of problems, so I'm guessing not. ;)

-- 
Grant Edwards               grant.b.edwards        Yow! ... the MYSTERIANS are
                                  at               in here with my CORDUROY
                              gmail.com            SOAP DISH!!