Execute complex shell commands within python and obtain the output.

[Cameron Simpson]

On 02Sep2019 13:20, Hongyi Zhao <hongyi.zhao at gmail.com> wrote:
>I try to execute some complex shell commands with in python and obtain
>the output, I tried the following method:
>
>For python 3.x:
>
>import subprocess
>cmd= some_complex_command_with_pipe_and_others
>ps = subprocess.Popen
>(cmd,shell=True,stdout=subprocess.PIPE,stderr=subprocess.STDOUT)
>output = ps.communicate()[0].decode('utf8')
>
>Is this the correct usage for this case?

It seems reasonable to me, but I would not use stderr=subprocess.STDOUT.  
Why did you do that? The stderr stream pretty much exists to avoid 
polluting stdout with error messages.

For really complex stuff you're often better writing a shell script and 
invoking that script, and avoiding shell=True. This has the advantages 
that:

(a) it avoids shell=True, a common source of accidents

(b) you don't have to embed shell punctuation in a Python string, which 
itself has punctuation

(c) you can run the script yourself by hand for testing purposes, 
outside the Python programme

(d) you're not restricted to the shell; the script might be in awk or 
any number of other languages

Finally, the .decode('utf8') assumes your locale is UTF8 based. It 
probably is, but if it isn't then you may get mojibake.

Cheers,
Cameron Simpson <cs at cskk.id.au>