[Tutor] accessing modules found throughout a package?

[Steven D'Aprano]

On Sat, Oct 17, 2015 at 08:20:14PM -0500, James Hartley wrote:
> In my current project, I am developing a package.  It makes sense to embed
> tests throughout the package's directory structure as they should be part
> of the package & its distribution.  It may raise eyebrows that I have tests
> sprinkled through various directories, but there are reasons for keeping
> some tests in the same proximity as the modules needed to import data into
> a database.  There are several directories importing data according to
> different constraints.

I'm sure that there are "reasons", but are they good reasons?

The package structure you show strikes me as odd. You have code under a 
"data" subdirectory, which seems deeply weird to me. Have you considered 
a more convention structure?


+-- package
    +-- __init__.py
    +-- module.py
    +-- subpackage
        +-- __init__.py
        +-- module.py
    +-- data
        +-- data.jpg
        +-- data.txt
    +-- tests
        +-- __init__.py
        +-- base_tests.py
        +-- more_tests.py
        +-- other_tests.py


which will correspond to Python imports like:

# Absolute imports
import package
import package.module
import package.subpackage
import package.subpackage.module
import package.tests
import package.tests.more_tests

which will work from your package's callers, and from within the package 
itself provided the top level directory can be found within Python's 
path. Within the package you can also use relative imports, see the 
docs for more detail.


So within (say) more_tests.py you should be able to say:

from . import base_tests

or

import package.tests.base_tests


You can get the path of the data subdirectory like so:

import package
path = os.path.join(package.__path__[0], "data")


At the very least, this conventional structure will avoid problems with 
the test modules, since they won't change location. Only your package 
modules themselves will change location during development.


-- 
Steve