organizing your scripts, with plenty of re-use

[Carl Banks]

On Oct 9, 4:37 pm, Buck <workithar... at gmail.com> wrote:
> We don't have any such convention. The production code is at a well-
> known path, but I need the code to be fully relocatable (cp -r /known/
> path ~/mydir) for testing purposes.
>
> Here's a scenario. A user does a cvs checkout into some arbitrary
> directory and sees this:
>
> project/
> +-- python/
>     +-- animals.py
>     +-- mammals/
>         +-- horse.py
>         +-- otter.py
>     +-- reptiles/
>         +-- gator.py
>         +-- newt.py
>     +-- misc/
>         +-- lungs.py
>         +-- swimming.py
>
> These are all runnable scripts that "just work" with no extra effort
> or knowlege, both in the testing scenario above, and for normal users
> that run it from some central location (maybe "/tools/mycompany/bin/
> mammals").
>
> The frustrating thing, for me, is that all these requirements are met
> if you leave the scripts in jumbled into a flat directory. As soon as
> you start organizing things, you need a good amount of boilerplate in
> each script to make things work anything like they did with the flat
> directory.
>
> I wouldn't mind having to write 'from mammals.horse import Horse' or
> 'from .animals import Animal' but it's not that simple.

Ok, presumably each of these files has something like the following
code inside them, since they can all be both imported and run as
scripts, right?

if __name__ == '__main__':
    main()

So follow these steps.

1. Remove this section from all of the files.  If the section contains
anything nontrivial, move the nontrivial stuff into your main()
function, or whatever function it is that call.  I'll just assume that
the function is called main.

2. In every directory and subdirectory below python, touch an
__init__.py file.  It can be empty.  You don't need one for the python
directory itself.

3. Change all your imports to package imports.  I recommend using
absolute imports only, but that's up to you.  "import otter" would
become "from mammals import otter", and so on.

4. Add the following basic script to your Python directory.  This is a
script, NOT A MODULE, so don't ever import it.  In fact, don't even
give it a py extension if you're not on Windows.  I'll call the script
"sa", for "simulate animals".  This script accepts the name of a
module to import on the command line, checks whether the top level
module is acceptable, imports the module, then calls the main function
in that module.  Enhance as required.

#!/usr/bin/python
import sys
allowed_top_levels = set(('animals','mammals','reptiles','misc'))
modname = sys.argv[1]
modpath = modname.split('.')
if modpath[0] not in allowed_top_levels:
    raise RuntimeError('invalid top level module specified')
mod = __import__(modname)
for pkgname in modpath[1:]:
    mod = getattr(mod,pkgname)
mod.main()  # or main(*sys.argv[2:]), or however you call it


5. Tell your users to stop entering commands like this:

$ ./python/mammal/otter.py


And start entering commands like this:

$ sa mammal.otter


With a little care this method will work fine, and will be installable
with cp -r.


Carl Banks