Package organization: where to put 'common' modules?

[Paul Boddie]

fortepianissimo wrote:
> Hm this doesn't work. Say I have the following directory structure:
>
> A
> |--- util
> |    |--- foo.py
> |
> |--- B
>      |--- bar.py
>
>
> And bar.py has this line
>
> from util import foo

This would only work with A in sys.path/PYTHONPATH. However...

> I then run
>
> python B/bar.py
>
> in directory A. Still got error
>
> ImportError: No module named util

Yes, Python does this - it puts the directory of bar.py (B in this
case) in sys.path, but not the directory in which you're sitting when
you run the program from the shell (A in this case). I always get round
this by running programs like this:

PYTHONPATH=. python B/bar.py

For me, this is a minor inconvenience since I know that ultimately the
imported packages will be installed and available via site-packages.

> A check of sys.path in bar.py when running from A revealed that
> sys.path has A/B, not A.
>
> Am I missing something here?

Well, I imagine that you want some kind of sys.path modification to
take place, and you can certainly do this in bar.py, taking care only
to do so in the "main program" section of the module - you don't
usually want sys.path magic going on when just importing from the bar
module.

However, my advice with regard to package structure and testing would
be to put common functionality either in modules below those that need
such functionality in the package structure, or in a separate hierarchy
in the package structure, to use absolute references to such packages
(A.util.foo) rather than relative ones (util.foo or stuff soon to be
permitted in Python 2.5), and to put your test programs outside the
package and have them use the package code like any other piece of
software.

Despite the presumed benefits to the upcoming relative import support
(less typing, perhaps), it's good to see that the existing import rules
will be tightened up with absolute importing being enforced. Here's an
example of import misbehaviour in today's Python:

---x
   |---y ...print "x"
   |---z ...import y.a # we want this to be absolute!

---y
   |---a ...print "y"

Now try and import x.z in Python 2.4! It won't even get to printing
"y".

Paul