Unit testing - one test class/method, or test class/class

[Edvard Majakari]

Hi,

I just found py.test[1] and converted a large unit test module to py.test
format (which is actually almost-no-format-at-all, but I won't get there
now). Having 348 test cases in the module and huge test classes, I started
to think about splitting classes. Basically you have at least three obvious
choises, if you are going for consistency in your test modules:

Choise a:

Create a single test class for the whole module to be tested, whether it
contains multiple classes or not.

...I dont think this method deserves closer inspection. It's probably rather
poor method to begin with. With py.test where no subclassing is required
(like in Python unittest, where you have to subclass unittest.TestCase)
you'd probably be better off with just writing a test method for each class
and each class method in the module.

Choise b:

Create a test class for each class in the module, plus one class for any
non-class methods defined in the module.

+ Feels clean, because each test class is mapped to one class in the module
+ It is rather easy to find all tests for given class
+ Relatively easy to create class skeleton automatically from test module
  and the other way round

- Test classes get huge easily
- Missing test methods are not very easy to find[2]
- A test method may depend on other tests in the same class

Choise c:

Create a test class for each non-class method and class method in the tested
module.

+ Test classes are small, easy to find all tests for given method
+ Helps in test isolation - having separate test class for single method
  makes tested class less dependent of any other methods/classes
+ Relatively easy to create test module from existing class (but then you
  are not doing TDD!) but not vice versa

- Large number of classes results in more overhead; more typing, probably
  requires subclassing because of common test class setup methods etc.

What do you think, any important points I'm missing?

Footnotes:
[1]  In reality, this is a secret plot to advertise py.test, see
     http://codespeak.net/py/current/doc/test.html

[2] However, this problem disappears if you start with writing your tests
    first: with TDD, you don't have untested methods, because you start by
    writing the tests first, and end up with a module that passes the tests
 

--
# Edvard Majakari		Software Engineer
# PGP PUBLIC KEY available    	Soli Deo Gloria!
One day, when he was naughty, Mr Bunnsy looked over the hedge into Farmer
Fred's field and it was full of fresh green lettuces. Mr Bunnsy, however, was
not full of lettuces. This did not seem fair.  --Mr Bunnsy has an adventure