Not finding user-installed module despite PYTHONPATH!

[Albert Hofkamp]

On Mon, 02 Dec 2002 19:04:11 GMT, Robin Munn <rmunn at pobox.com> wrote:
> Albert Hofkamp <hat at se-46.wpa.wtb.tue.nl> wrote:
>> On Fri, 29 Nov 2002 19:08:42 GMT, Robin Munn <rmunn at pobox.com> wrote:
>>> Albert Hofkamp <hat at se-46.wpa.wtb.tue.nl> wrote:
>>>> 
>>>> I want to run an interactive Python module through ssh. The basic solution is
>>>> something like
>>>> 
>>>> ssh -x remote_machine 'python -c "import mymodule ; mymodule.run()"'
>>> 
>>> [snip attempts to get PYTHONPATH set]
>>> 
>>> Does something like this work?
>>> 
>>> ssh -x remote_machine 'PYTHONPATH=/path/to/module python -c "import mymodule ; mymodule.run()"'
>>> 
>>> (If that line wraps, it should be all on one line).
>> 
>> (it didn't)
>> At first sight, there seems to be a ; missing before 'python'. Also,
>> some shells may need something like 'export' or 'setenv' at the beginning.
>> (I will try this option, but near the end of the week or later).
> 
> No, there is no semicolon missing. This is standard syntax for
> temporarily setting an environment variable under the Bourne shell (or
> its descendants, such as bash). The generalized syntax is:
> 
>     VARIABLE=value program arguments
> 
> Or, if you want to set several environment variables:
> 
>     VAR1=val1 VAR2=val2 VAR3=val3 program arguments
> 
> If you have a copy of Unix Power Tools, see recipe 6.10 (page 100 in the
> second edition, might be a different page number in the third edition).
> 
> However, this only works with the Bourne shell. See below for a more
> generalized solution.
> 
>> 
>>> It seems to me that this might be a better solution than trying to set
>> 
>> I am a bit afraid what assumptions you make above w.r.t. the used shell
>> (e.g. must I use export or setenv?).
>> 
>> The overall idea was
>> a) make as little assumptions about the remote system as possible
>> b) make it easy to maintain.
>> c) make it easy to explain to non-unix experts.
>> 
>> I think I cannot comply to all these conditions with either
>> ".ssh/enivronment" or with "PYTHONPATH="
>> 
>> What worries me most is c). The entire problem is in my view a Unix problem
>> which should be dealt with by configuring shells, etc. It is not part of
>> the module functionality, yet it looks like I will need to spend at least
>> half a page of text to this problem :-(
> 
> It seems to me that configuring shells, etc. violates requirement a). I
> was explictly trying to make as little assumptions *and* modifications
> about the remote system as possible. In other words, the environment on
> the remote system should *not* need to be touched. Therefore the
> appropriate environment variable must be set in a temporary way.

I agree, yet I reached the opposite conclusion....

What I realize now is that I assumed a working environment was set up at
the remote end (i.e. after "ssh remote_machine", the user can do work at
the remote machine).
This assumption changes the meaning of "a)" quite a bit. If the user can
work at the remote machine, he has machine-specific settings there, such as
PYTHONPATH. Duplicating those settings because the user runs a non-standard
shell creates a consistency problem between both machines.

Your assumption of not knowing/assuming anything (and overriding the
setting to make sure it works) also has its merits, particularly if the
remote machine has no correctly setup environment.


Since both types of machines exist in the world, I will probably adapt the
documentation to include both options. The user can then chose what he/she
likes best.

> is going to be running the Bourne shell. Here's a more generalized
> solution that should apply even if the machine is running the C shell:
> 
> ssh -x remote_machine 'env PYTHONPATH=/path/to/module python -c "import mymodule ; mymodule.run()"'

Brilliant.

> The only assumption this makes is that the 'env' program is available.
> This program *should* be standard on any Unix made in the past fifteen
> years or so -- someone please correct me if I'm wrong.

Isn't it ironic?
We want to have different shells, which Unix allows. Next, it becomes quite
impossible to do something shell-independant, so we create env as a kind of
meta-shell to resolve such matters.
Maybe we should all use env as shell :-)

> module is only found in one place in one source code file, making for
> easy code updates if the path ever changes. You could even flag it with
> a comment:
> 
>     # ***** CHANGEME: Change the path below if the module ever moves *****
>     ssh -x remotemachine ...
> 
> Replace 'CHANGEME' with whatever string makes sense to you and that you
> would remember when trying to grep a source file for necessary
> changes...

Actually, the string to execute is a string parameter of the constructor of
a Python class. Its value is not part of my module code, it is in
user-supplied code.
The documentation of the module should be extended to include this example.

> As for the need to explain your code to non-Unix experts, do you want to
> be giving them a crash course on environment variables and the many
> subtleties about using them (like how the environment passes from parent
> process to child process but never in reverse, and the implications of
> this on shell scripts)? Or do you just want to say "The python program
> needs to have the PYTHONPATH environment variable set correctly to be
> able to find the correct module; the call to 'env' accomplishes this."

The latter would (obviously) be preferable, together with the example ssh
stringr. This would be enough for me (after some reading on env, I would be
able to figure out what is happening). It would probably also be enough for
many others here, except that they wouldn't have a clue what they are doing
(they simply copy/paste/adapt the example string, until it does not report
any error). If it then produces seemingly correct results, they are happy,
otherwise they are lost and end up at my desk.

There is however also a group in between. This group will need some
pointers (such as a discussion about environment vars in a book) to
understand and fix the problem.
(your reference to the unix power tools book may be good here, I will read
that, and also do some more reading on ssh to check whether they discuss
the problem).

> 
> Anyway, I hope this helps.

It does, thanks very much !!


Albert
-- 
Unlike popular belief, the .doc format is not an open publically available format.