Understanding relative imports in package - and running pytest with relative imports?

[Devin Jeanpierre]

On Sun, Nov 24, 2013 at 5:30 PM, Victor Hooi <victorhooi at gmail.com> wrote:
> The advice seems to be either to run it from the parent directory of furniture with:
>
>     python -m furniture.chair.build_chair

Yes. More pedantically, run it from somewhere such that the furniture
package is importable. For example, if you install furniture, then
it's accessible everywhere.

> However, I don't see having a separate single main.py outside my package would work with keeping my code tidy/organised, and or how it'd work with the other files (config.yaml, or create_table.sql) which are associated with each script?

The other files are contained within the package. As long as
build_table looks in the right places, they'll still be accessible.
(e.g. http://docs.python.org/2/library/pkgutil#pkgutil.get_data ).

> A third way I thought of way just to create a setup.py and install the package into site-packages - and then everything will work? However, I don't think that solves my problem of understanding how things work, or getting my directory structure right.

No, installing furniture won't change anything about how you need to
run it. Either you run a separate script that imports it and runs the
right main function, or you use python -m furniture.foo.bar

> Although apparently running a script inside a package is an anti-pattern? (https://mail.python.org/pipermail/python-3000/2007-April/006793.html)

It's not an antipattern, it's just plain broken. Some things stop
working as you'd expect, like exception handling.

> How would you guys organise the code above?

If I really cared about the command line interface, I would probably
reorganize something like this:

furniture/
 - __init__.py
 - __main__.py
 - common.py
 * chair/
   - __init__.py
   - build.py
   - create.sql
   - config.yaml
 * furniture/
   - __init__.py
   - build.py
   - create.sql
   - config.yaml

And then create chairs by running `python -m furniture build chair`.
furniture/__main__.py would dynamically import furniture.chair.build
and run furniture.chair.build.main().

There would also be an executable I might install separately,
furniture, the contents of which are "from furniture import __main__;
__main__.main()"

> Also, if I have tests (say with pyttest), inside furniture/table/tests/test_table.py, how would I run these as well? If I run py.test from there, I get the same:
>
>     $ py.test
>      ....
>      from ..table.build_table import Table
>      E   ValueError: Attempted relative import in non-package
>     ....
>
> (Above is just an extract).
>
> Assuming I use pytest, where should my tests be in the directory structure, and how should I be running them?

As far as I understand py.test and nose, they are designed such that
you don't put your tests as subpackages. If you want to put your tests
as subpackages, use a different unit testing framework, like unittest.
Then you can just do `python -m unittest discover .`

If you do want to use py.test (or nose), then either there is some
flag you need to pass in that I'm not aware of, or your directory
structure should be:

my-project/
 - README
 - LICENSE
 - setup.py
 * furniture/
    ...
 * tests/
   - test_chair.py
   - test_common.py
   - test_table.py

Note that tests is not a package either. py.test and nose work by
loading files, NOT modules. So they won't work with loading test
submodules in a package.

For this and other reasons, I personally stay away from nose and py.test.

-- Devin