Popen Question

[Chris Torek]

In article <891a9a80-c30d-4415-ac81-bddd0b564fa6 at g13g2000yqj.googlegroups.com>
moogyd  <moogyd at yahoo.co.uk> wrote:
>[sde:staff at lbux03 ~]$ python
>Python 2.6 (r26:66714, Feb 21 2009, 02:16:04)
>[GCC 4.3.2 [gcc-4_3-branch revision 141291]] on linux2
>Type "help", "copyright", "credits" or "license" for more information.
>>>> import os, subprocess
>>>> os.environ['MYVAR'] = "myval"
>>>> p = subprocess.Popen(['echo', '$MYVAR'],shell=True)

Alain Ketterlin has already explained these to some extent.
Here is a bit more.

This runs, underneath:

    ['/bin/sh', '-c', 'echo', '$MYVAR']

(with arguments expressed as a Python list).  /bin/sh takes the
string after '-c' as a command, and the remaining argument(s) if
any are assigned to positional parameters ($0, $1, etc).

If you replace the command with something a little more explicit,
you can see this:

    >>> p = subprocess.Popen(
    ...    [r'echo \$0=$0 \$1=$1', 'arg0', '$MYVAR'], shell=True)
    >>> $0=arg0 $1=$MYVAR
    p.wait()
    0
    >>> 

(I like to call p.communicate() or p.wait(), although p.communicate()
is pretty much a no-op if you have not done any redirecting.  Note that
p.communicate() does a p.wait() for you.)

>>>> p = subprocess.Popen(['echo', '$MYVAR'])
>>>> $MYVAR

This time, as Alain noted, the shell does not get involved so no
variable expansion occurs.  However, you could do it yourself:

    >>> p = subprocess.Popen(['echo', os.environ['MYVAR']])
    >>> myval
    p.wait()
    0
    >>> 

>>>> p = subprocess.Popen('echo $MYVAR',shell=True)
>>>> myval

(here /bin/sh does the expansion, because you invoked it)

>>>> p = subprocess.Popen('echo $MYVAR')
>Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/usr/lib64/python2.6/subprocess.py", line 595, in __init__
>    errread, errwrite)
>  File "/usr/lib64/python2.6/subprocess.py", line 1106, in
>_execute_child
>    raise child_exception
>OSError: [Errno 2] No such file or directory

This attempted to run the executable named 'echo $MYVAR'.  It did
not exist so the underlying exec (after the fork) failed.  The
exception was passed back to the subprocess module, which raised
it in the parent for you to see.

If you were to create an executable named 'echo $MYVAR' (including
the blank and dollar sign) somewhere in your path (or use an explicit
path to it), it would run.  I will also capture the actual output
this time:

    $ cat '/tmp/echo $MYVAR'
    #! /usr/bin/awk NR>1{print}
    this is a self-printing file
    anything after the first line has NR > 1, so gets printed
    $ chmod +x '/tmp/echo $MYVAR'
    $ python
    Python 2.5.1 (r251:54863, Feb  6 2009, 19:02:12) 
    [GCC 4.0.1 (Apple Inc. build 5465)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import subprocess
    >>> p = subprocess.Popen('/tmp/echo $MYVAR', stdout=subprocess.PIPE)
    >>> print p.communicate()[0]
    this is a self-printing file
    anything after the first line has NR > 1, so gets printed

    >>> p.returncode
    0
    >>> 

Incidentally, fun with #!: you can make self-renaming scripts:

    sh-3.2$ echo '#! /bin/mv' > /tmp/selfmove; chmod +x /tmp/selfmove
    sh-3.2$ ls /tmp/*move*
    /tmp/selfmove
    sh-3.2$ /tmp/selfmove /tmp/I_moved
    sh-3.2$ ls /tmp/*move*
    /tmp/I_moved
    sh-3.2$

or even self-removing scripts:

    sh-3.2$ echo '#! /bin/rm' > /tmp/rmme; chmod +x /tmp/rmme
    sh-3.2$ /tmp/rmme
    sh-3.2$ /tmp/rmme
    sh: /tmp/rmme: No such file or directory

(nothing to do with python, just the way #! interpreter lines work).
-- 
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W)  +1 801 277 2603
email: gmail (figure it out)      http://web.torek.net/torek/index.html