Correctly reading stdout/stderr from subprocess

[Maric Michaud]

Le Mercredi 14 Juin 2006 00:20, Christoph Haas a écrit :
> Evening,
>
> I'm having trouble with running a process through Python 2.4's
> subprocess module. Example code:
>
> ========================================================
> def run(command):
>    run = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE,
> stderr=subprocess.PIPE)
>
>    # Wait for the process to return
>    returncode = run.wait()
>    stdout = run.stdout.readlines()
>    stderr = run.stderr.readlines()
>
>    # Strip newlines at the end of each line
>    stdout = [line.rstrip('\n') for line in stdout]
>    stderr = [line.rstrip('\n') for line in stderr]
>
>    return returncode,stdout,stderr
> ========================================================
>
> Unfortunately this def fails when it runs programs that do a lot of
> output. I believe this problem has also been described in Bug #1162428.
> It says: "You must read away the data before wait() on the process."
> Easier said than done. If I put the readlines() call before the
> run.wait() then the process hangs, too, waiting for further lines.
>

This because you can't predict which of the two output will block, either 
merge the outpus with a popen4 or use communicate like below or use a 
threaded version (much more complicated).

> Then I read about the communicate() call which handles both
> stdout/stderr reading and also waiting for the process to end.
> Unfortunately it returned the output char-wise instead of line-wise. I
> could certainly re-join the lines. But somehow I feel I'm missing
> something simple.
>
> Any ideas?
>

def run2(command):

   import subprocess

   run = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, 
stderr=subprocess.PIPE)

   # Wait for the process to return
   out, err = [ e.splitlines() for e in run.communicate() ]

   return run.returncode, out, err

-- 
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097