[Tutor] Best way to get root project directory in module search path

[Cameron Simpson]

On 26Aug2017 21:27, boB Stepp <robertvstepp at gmail.com> wrote:
>I have a project structure where I am developing various programs,
>some in Pinnacle HotScript language, some in Perl, some in Python,
>some shell scripts.  For the purpose of this question, my simplified
>development file structure is:
>
>~/_dev_Scripts
>    /ScriptMenuSystem
>        __init__.py
>        test.py
>
>    /py_utilities
>        __init__.py
>        test.py
>
>When I am fully happy (Not really! ~(:>)) ) with everything I will
>copy everything pertinent from "~_dev_Scripts" into a location like
>"/usr/local/.../Scripts/boB_Scripts".  So all code must be able to
>handle any change in location from where I am developing it.  On to my
>difficulties ...
>
>ScriptMenuSystem/test.py:
>
>-------------------------------------------------
>#!/usr/bin/env python
>
>import sys
>sys.path.append('..')
>
>import py_utilities.test as t
>
>t.test_print()

The trouble with this specific approach is that '..' relies on your _working_ 
directory being above ScriptMenuSystem i.e. the directory you shell's in; '..' 
is not related to the path to the test.py file. What yu want for this is the 
actual path to the code i.e. __file__.  Eg:

  sys.path.append(dirname(dirname(__file__)))

but that requires your module to know its own location within your library i.e.  
that it is .../dev_scripts/ScriptMenuSystem/test.py, and therefore that you 
want .../dev_scripts. Fragile. (And __file__ doesn't work in some more arcane 
ways of distributing modules, such as in zip files but let's not go there, not 
least because I've never done that myself).

The usual approach is to burden the user of the module. They're using it: 
ideally they have "installed" it by putting it somewhere sensible. So in a 
sense you should be able to rely on $PYTHONPATH already being set up.

Think about the end game: supposing people are using your modules in some 
larger system (eg a virtualenv). Do they really even _want_ you mucking about 
with their $PYTHONPATH or sys.path or, really, anything that will affect how 
modules are looked up?

Personally, I rely on $PYTHONPATH being right when I'm called. If someone's 
"installed" your module (eg by putting it in site-packages or wherever) the 
path is already set up. For my development purposes I invoke my code via a 
"dev" shell function which prehacks $PYTHONPATH to have my local lib directory 
at the start of $PYTHONPATH. Again, the module itself can then trust that 
things are as they should be. Which is as it should be.

Cheers,
Cameron Simpson <cs at cskk.id.au> (formerly cs at zip.com.au)