GCC process not working as expected when called in Python (3.1.2) subprocess-shell, but OK otherwise

[Chris Rebert]

On Wed, Oct 13, 2010 at 7:06 PM, Kingsley Turner
<kingsley.turner at openfieldcommunications.com> wrote:
>  Hi,
>
> I'm using GCC as a pre-processor for a C-like language (EDDL) to handle all
> the includes, macros, etc. producing a single source file for another
> compiler.  My python code massages the inputs (which arrive in a .zip file),
> then calls GCC.
>
> I have a problem where if I call GCC from my python script some of the
> #defines are not processed in the output.  However if I paste the exact same
> GCC command-line into a shell, I get a correct output.
>
> I'm calling GCC in this manner:
>
>    ### Execute GCC, keep stdout & stderr
>    err_out = open(error_filename,"wb")
>    process = subprocess.Popen(gcc_command, stderr=err_out, bufsize=81920,
> cwd=global_options['tmp'])
>    gcc_exit_code = process.wait()
>    log("GCC Exit Code %u" % (gcc_exit_code))
>    err_out.close()
>
> where gcc_command is:
>
>    "/usr/bin/gcc -I /tmp/dd-compile_1286930109.99 -I /home/foo/eddl-includes
> -D__TOKVER__=600 -ansi -nostdinc -v -x c -E -o
> /tmp/dd-compile_1286930109.99/11130201.ddl.OUT
> /tmp/dd-compile_1286930109.99/11130201.ddl"

Quoting http://docs.python.org/library/subprocess.html#subprocess.Popen
, emphasis mine, and keeping in mind that you're passing gcc_command
as the "args" argument to Popen's constructor:

"On Unix, with shell=False (default): [...] args should normally be
**a sequence**. If a string is specified for args, it will be used as
the name or path of the program to execute; ***this will only work if
the program is being given no arguments***."

Clearly you are trying to run GCC with arguments, hence your problem.
Either pass shell=True to Popen(), or, preferably, change gcc_command
to a properly tokenized list of strings; see the docs section I linked
to, it gives an example of how to do so.

Cheers,
Chris
--
Lesson: Read the `subprocess` docs. They've gotten better.
http://blog.rebertia.com