[Tutor] Best way to setup Unit Testing?

[Oscar Benjamin]

On 7 July 2013 21:16, Srinivas Nyayapati <shireenrao at gmail.com> wrote:
> I am tryng to figure out the best way to do Unit Testing for all my projects
> going forward.

Are you the only one using these applications or are you intending
that other people would install them? Do you want other people to run
the tests on their machines when they install? Will other people also
work on the development of these projects and will they also need to
run the tests?

> I am using the unittest framework. For execution I am
> executing the test cases by either directly using python or by using the
> nose test runner. I found the following 3 types of how this can be setup.
>
> Type 1
> ======
>
> The unit tests are located in the same package as the module being tested.
> This is the easiest to implement and run. I don't have to do anything
> special to run the tests. The directory structure is like this
>
> Projects/
>     MyApp/
>         __init__.py
>         myapp.py
>         test_myapp.py

The (possible) advantage of this setup is that a user can run tests on
the application while it is installed using:

$ python -m MyApp.test_myapp

The above could get messy if the number of submodules increases though.

>
> Simply run the test from within the MyApp folder as
>
> % python test_myapp.py
> or use the nose test runner
>
> % nosetests
>
> Type 2
> ======
>
> Here the unit tests are located in its own tests folder in the package you
> want to test. Running the unit tests from inside the tests folder won't work
> as it worked in Type 1. That's because the myapp module can not be resolved.
> Here is how the directory structure looks like
>
> Projects/
>     MyApp/
>         __init__.py
>         myapp.py
>         tests/
>             __init__.py
>             test_myapp.py

This gives a nice clean distinction between your main code and your
tests. It is also useful if you don't want to install the tests on the
target machines since you can just exclude the tests package in your
setup.py when creating an sdist.

If you do include the tests and add a __main__.py in the tests folder
then users can still run tests with

python -m MyApp.tests

> It will work if you ran it from inside the package folder and gave the
> complete path to the test file as
>
> % python tests/test_myapp.py
> It will also work if you used the nose test runner from inside MyApp.
>
> % nosetests
> It will also work from inside the tests folder if you added MyApp to your
> PYTHONPATH, but that is an option I dont like. I don't want to keep changing
> my PYTHONPATH.

You can always just do that in a script. I would normally have a
Makefile and invoke tests with

$ make test

Inside the Makefile you can do

test:
    PYTHONPATH=src nosetest

Then you don't need to fiddle around with PYTHONPATH manually. My
preference is that 'make test' should be run from the top-level
directory of the project (if you're using version control then that's
the main folder that gets created in a checkout/clone). Someone with
the code should be able to do

$ cd SomeProject
$ make test
........ 121 out of 121 test passed.

>
> Type 3
> ======
>
> This is where the tests folder is on the same level as the package folder.
> You can not run the test scripts from within the tests folder as again it
> will not be able to resolve the myapp package. Here is how the directory
> structure looks like
>
> Projects/
>     MyApp/
>         __init__.py
>         myapp.py
>     tests/
>         __init__.py
>         test_myapp.py

In this setup you are not intending for the tests to be installed on
the target machines and you have a clear separation between your main
code and your tests.

>
> You can run the tests from within the MyApp package by referencing the tests
> folder as
>
> % python ../tests/test_myapp.py
>
> As you are running the test from the package, the myapp module is
> automatically in your PYTHONPATH. The other way of doing this would be by
> putting your package in your PYTHONPATH, which again as I mentioned earlier
> I don't like to do.
>
> What is the best way of doing this? I would really appreciate to know how
> the experts are doing this. Is there anything I need to consider before I
> start using test suites.

Whatever you do just use a script or a Makefile or something so that
invoking the tests is a simple one-liner e.g.

$ make test
$ python runtests.py
$ ./test.sh

or whatever suits you. If you need to fiddle with PYTHONPATH do it in
that script, rather than manually.


Oscar