Do subprocess.PIPE and subprocess.STDOUT sametime

[jak]

Horst Koiner ha scritto:
> Hi @all,
> i'm running a program which is still in development with subprocess.run (Python version 3.10), further i need to capture the output of the program in a python variable. The program itself runs about 2 minutes, but it can also freeze in case of new bugs.
> 
> For production i run the program with stdout=subprocess.PIPE and i can fetch than the output later. For just testing if the program works, i run with stdout=subprocess.STDOUT and I see all program output on the console, but my program afterwards crashes since there is nothing captured in the python variable. So I think I need to have the functionality of subprocess.PIPE and subprcess.STDOUT sametime.
> 
> What I tried until now:
> 1. Poll the the output and use Popen instead:
> 
> # Start the subprocess
> process = subprocess.Popen(['./test.sh'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> 
> captured_output = b''
> process_running = True
> while process_running:
>      process_running = (process.poll() is not None)
>      for pipe in [ process.stdout, process.stderr ]:
>          while line := pipe.readline():
>              print(line)
>              captured_output += line
> 
> print(captured_output)
> return_code = process.returncode
> 
> => But this is discouraged by the python doc, since it says that polling this way is prone to deadlocks. Instead it proposes the use of the communicate() function.
> 
> 2. Use communicate() with timeout.
> => This works not at all since when the timeout occurs an exception is thrown and communicate returns at all.
> 
> 3. Use threading instead
> => For being that simple and universal like subprocess you will more or less reimplement subprocess with threading, like its done in subprocess.py. Just for a debug output the effort is much to high.
> 
> #######################################################
> Do you have further ideas for implementing such a behavior?
> Do you think that a feature request should be done of I'm omitting something obvious?
> 
> Thanks you in advance for your suggestions,
> Horst.
> 

I agree with @'thomas Passin' but I solved in a different way, I made 
the Readline() not blocking. even if I believe his idea better than
mine:

os.set_blocking(process.stdout.fileno(), False)
os.set_blocking(process.stderr.fileno(), False)