[Tutor] run local script on a remote machine

[Cameron Simpson]

On 26Oct2016 10:44, Alex Kleider <akleider at sonic.net> wrote:
[... snipped experiment.py and the demo call.sh script ...]
>3:
>#!/usr/bin/env python3
>#
># file: call.py
>
>import os
>import shlex
>import subprocess
>
>script = "/home/alex/Py/BackUp/Sandbox/Scripted/experiment.py"

This is fine.

>if os.path.isfile(script):
>    print("File exists on local machine.")
>else:
>    print("No such file.")

This is fine.

>command = (
>"ssh -p22 alex at 10.10.10.10 python3 -u - one two three < {}"
>    .format(script))
>ret = subprocess.call(shlex.split(command))

This is not fine.

There are a few things wrong here.

First, as a maytter of practice you should _never_ construct command strings as 
you are doing here: embedding an arbitrary filename directly in a string that 
will be evaluated by a command parser. This applies equally to shell commands 
such as yours or to SQL or anything else where you're preparing text to be 
parsed. Supposing your filename has shell punctuation in it, even the lowly 
space character? The outcome would not be what you intended. You can see where 
I'm going here I'm sure. Read more here (SQL specific, though the principle is 
general):

  http://bobby-tables.com/

The second problem, and the one causing your immediate issue, is the use of 
shlex.split. This is _not_ a full shell parser. It is a convenience function 
that recognises common shell quoting syntax to be used when writing little 
minilanguages of your own - it gets you a preimplemented quoting parser you can 
then let your users access to mark off strings.

Your command looks like this, roughly:

  ssh -p22 ..blah blah blah... < /path/to/script.py

All shlex.split is going to do with this is to break it into separate strings:

    ssh
    -p22
    ..blah
    blah
    blah...
    <
    /path/to/script.py

They're just strings. No redirection recognition or implementation is happening 
here. Then when you call subprocess.call:

    ret = subprocess.call(shlex.split(command))

you're passing a list of those strings as the first argument. subprocess.call 
and its partner Popen have two modes for their first argument: if it is a 
string then it will be passed to the system shell, otherwise it is executed 
directly without being handled by a shell. Effectively the first form takes:

    subprocess.call(some_string)

and runs is as:

    subprocess.call(['/bin/sh', '-c', some_string])

You're using the second form, and that is by far the better thing to do, so 
good there. _However_, you're effectively invoking ssh with no redirections; 
instead your passing the strings '<' and '/path/to/script.py' as arguments to 
ssh.

What you really want to do is this (untested):

  with open(script) as scfp:
    ret = subprocess.call(['ssh', '-p22', 'alex at 10.10.10.10',
                           'python3', '-u', '-',
                           'one', 'two', 'three'],
                          stdin=scfp)

which arranges to attach an open file reading from your script to the ssh 
subprocess you are invoking. Note that it does _not_ pass the script pathname 
itself as an argument to ssh. Neither does your shell script.

Now for the confusing bit: what _was_ your program really doing? Well, ssh 
concatenates its command arguments:

  python3 -u - one two three

together and passes them to the far end, which hands them to the shell! So 
effectively, like the string form of subprocess.call, ssh itself effectively 
invokes:

  sh -c 'python3 -u - one two three'

at the far end. So you're going to need shell quotes in that string if you ever 
pass something slightly tricky, or something arbitrary. But you're not yet.

_However_, the original form of you program was passing _these_ strings as 
command arguments to ssh:

    python3 -u - one two three < /home/alex/Py/BackUp/Sandbox/Scripted/experiment.py

so ssh is invoking this:

    /bin/sh -c 'python3 -u - one two three < /home/alex/Py/BackUp/Sandbox/Scripted/experiment.py'

at the far end. (Those are all on one line, BTW, in case your email reader 
folds things up.)

So you might think: but then the shell _will_ see the "<" redirection! But of 
course that is the shell on the remote machine, and your script isn't on that 
machine, so the shell emits the error message you saw.

Hoping this clarifies what's going on and how to go forward.

Please feel free to ask any questions that occur.

Cheers,
Cameron Simpson <cs at zip.com.au>

Hofstadter's Law: It always takes longer than you expect, even when you take
into account Hofstadter's Law.
- Douglas Hosfstadter, Godel, Escher, Bach: an Eternal Golden Braid