[Tutor] Best way to setup Unit Testing?

[Srinivas Nyayapati]

On Jul 10, 2013, at 6:48 AM, Oscar Benjamin <oscar.j.benjamin at gmail.com> wrote:

> On 7 July 2013 21:16, Srinivas Nyayapati <shireenrao at gmail.com> wrote:
>> I am tryng to figure out the best way to do Unit Testing for all my projects
>> going forward.
>
> Are you the only one using these applications or are you intending
> that other people would install them? Do you want other people to run
> the tests on their machines when they install? Will other people also
> work on the development of these projects and will they also need to
> run the tests?

So far I am the only one. But in case I wanted to share my projects (I
hope!) I wanted to make sure the other party can run tests easily.

>> I am using the unittest framework. For execution I am
>> executing the test cases by either directly using python or by using the
>> nose test runner. I found the following 3 types of how this can be setup.
>>
>> Type 1
>> ======
>>
>> The unit tests are located in the same package as the module being tested.
>> This is the easiest to implement and run. I don't have to do anything
>> special to run the tests. The directory structure is like this
>>
>> Projects/
>>    MyApp/
>>        __init__.py
>>        myapp.py
>>        test_myapp.py
>
> The (possible) advantage of this setup is that a user can run tests on
> the application while it is installed using:
>
> $ python -m MyApp.test_myapp
>
> The above could get messy if the number of submodules increases though.

I agree and will not be doing this.

>> Simply run the test from within the MyApp folder as
>>
>> % python test_myapp.py
>> or use the nose test runner
>>
>> % nosetests
>>
>> Type 2
>> ======
>>
>> Here the unit tests are located in its own tests folder in the package you
>> want to test. Running the unit tests from inside the tests folder won't work
>> as it worked in Type 1. That's because the myapp module can not be resolved.
>> Here is how the directory structure looks like
>>
>> Projects/
>>    MyApp/
>>        __init__.py
>>        myapp.py
>>        tests/
>>            __init__.py
>>            test_myapp.py
>
> This gives a nice clean distinction between your main code and your
> tests. It is also useful if you don't want to install the tests on the
> target machines since you can just exclude the tests package in your
> setup.py when creating an sdist.
>
> If you do include the tests and add a __main__.py in the tests folder
> then users can still run tests with
>
> python -m MyApp.tests

Love this idea! Would you know how I would run the test_*.py scripts
from __main__.py


>> It will work if you ran it from inside the package folder and gave the
>> complete path to the test file as
>>
>> % python tests/test_myapp.py
>> It will also work if you used the nose test runner from inside MyApp.
>>
>> % nosetests
>> It will also work from inside the tests folder if you added MyApp to your
>> PYTHONPATH, but that is an option I dont like. I don't want to keep changing
>> my PYTHONPATH.
>
> You can always just do that in a script. I would normally have a
> Makefile and invoke tests with
>
> $ make test
>
> Inside the Makefile you can do
>
> test:
>    PYTHONPATH=src nosetest
>
> Then you don't need to fiddle around with PYTHONPATH manually. My
> preference is that 'make test' should be run from the top-level
> directory of the project (if you're using version control then that's
> the main folder that gets created in a checkout/clone). Someone with
> the code should be able to do
>
> $ cd SomeProject
> $ make test
> ........ 121 out of 121 test passed.

Another really neat idea! I am going to implement make files to do my
tests.. Came across an issue while implementing this. The problem was
my test target would not run as it said it was upto date. Adding the
following took care of that.

.PHONY: tests


>>
>> Type 3
>> ======
>>
>> This is where the tests folder is on the same level as the package folder.
>> You can not run the test scripts from within the tests folder as again it
>> will not be able to resolve the myapp package. Here is how the directory
>> structure looks like
>>
>> Projects/
>>    MyApp/
>>        __init__.py
>>        myapp.py
>>    tests/
>>        __init__.py
>>        test_myapp.py
>
> In this setup you are not intending for the tests to be installed on
> the target machines and you have a clear separation between your main
> code and your tests.

I get the difference between the different styles now.

>>
>> You can run the tests from within the MyApp package by referencing the tests
>> folder as
>>
>> % python ../tests/test_myapp.py
>>
>> As you are running the test from the package, the myapp module is
>> automatically in your PYTHONPATH. The other way of doing this would be by
>> putting your package in your PYTHONPATH, which again as I mentioned earlier
>> I don't like to do.
>>
>> What is the best way of doing this? I would really appreciate to know how
>> the experts are doing this. Is there anything I need to consider before I
>> start using test suites.
>
> Whatever you do just use a script or a Makefile or something so that
> invoking the tests is a simple one-liner e.g.
>
> $ make test
> $ python runtests.py
> $ ./test.sh
>
> or whatever suits you. If you need to fiddle with PYTHONPATH do it in
> that script, rather than manually.

Thank you for your detailed answer. Learnt something new today.

Srini