Call a shell command from Python

[Steve D'Aprano]

On Tue, 1 Nov 2016 04:00 pm, Wildman wrote:

> You are correct about that but, in this case grep never "sees" the '$'
> sign.  Bash expands $USER to the actual user name beforehand.  If you
> are on a Linux system, enter this into a terminal to illustrate:
> 
> sudo grep ^$USER\: /etc/shadow

Bash is not involved here. Python is calling grep directly.


You don't have to believe us, you can test this yourself. Create a simple
text file with a single line containing your username, and a simple Python
script that calls grep as you have been:


[steve at ando ~]$ echo $USER
steve
[steve at ando ~]$ cat foo.txt
blah blah steve blah blah
[steve at ando ~]$ cat greptest.py
import subprocess
cmdlist = ['grep', '$USER', 'foo.txt']
p = subprocess.Popen(cmdlist, stdout=subprocess.PIPE,stderr=subprocess.PIPE)
line, err = p.communicate()
print err, line

[steve at ando ~]$ python2.7 greptest.py

[steve at ando ~]$



So there you have it: categorical proof that bash does not expand the
string '$USER'. It cannot: bash is not involved in the subprocess call.
Python calls grep directly.

If you want to expand the '$USER' string from Python, do it yourself:


py> import os
py> os.environ['USER']
'steve'


> If the user name is ben then grep would see this:
> 
> grep ben\: /etc/shadow

If would, if you called grep from bash. But you didn't.


>>> > Maybe you are expecting Bash to be involved somehow (and so “$USER”
>>> > will be substituted by Bash with some other value). That's not what
>>> > happens.
>>>
>>> No, the shell is already running.
>> 
>> I don't know what you mean by this. If you mean that some *other*
>> instances of the shell ar running: that isn't relevant to how your
>> Python program invokes a subprocess.
> 
> I simply meant that the script is run from a terminal.

That's irrelevant. Just because the script is running from a terminal
doesn't mean that the shell can peer deep inside each and every process and
magically apply the shell's string expansion rules.





-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.