Unit Testing Techniques

[Matthew Fitzgibbons]

I'm by no means a testing expert, but I'll take a crack at it.

Casey McGinty wrote:
> I'm familiar with the unittest module in Python, however I'm hoping 
> someone can point me to some examples of more advanced usages of the 
> framework. For example:
> 
> 1. Using the framework to test a package with nested sub-packages and 
> modules without having to hard code the location/name of each test module.
I've never run into this.

> 2. Testing class/methods that depend on file system directory 
> structures, file data, or values read from system hardware.
Rule of thumb: always separate software from hardware. Write mock 
classes or functions that do your hardware/file access that always 
return known data (but remember to test for alpha and beta errors--make 
sure both valid and invalid data are handled correctly). That way you 
can test the client code that is accessing the hardware.

Testing the actual hardware/file access code can get more interesting. 
If you're just reading files, your test suite should write the test 
files in setUp, try to read them in the test* methods, then clean up in 
tearDown. Again, remember to test both good and bad files.

In my shop, we do both hardware and software, and testing the comms 
between them can be a real pain. We've done lots of interesting things 
like wiring faults into the hardware with switches to connect and 
disconnect them, recording command sequences, etc., but I expect this is 
beyond the scope of what you're interested in. Just keep in mind that 
testing a chunk of code requires at least as much creativity as writing 
the code in the first place.

> 3. Testing class/methods that require simulating functionality from 
> imported modules.
You can do neat things like this:

import unittest
import mymodule

def my_mock_function():
	"""Returns known values."""


class MyTest(unittest.TestCase):
	def setUp(self):
		self._orig_function = mymodule.function
		mymodule.function = my_mock_function

	def tearDown(self):
		# remember to restore the original function
		# unittest doesn't re-import modules
		mymodule.function = self._orig_function

	def test1(self):
		"""Test some code that uses mymodule."""

# etc...

The dynamic nature of Python makes this sort of thing much easier than 
other languages.

> 4. Testing graphical interfaces and associated signal callback functions.
Again, there are lots of strategies you can use; the specifics depend on 
you design and the toolkit.

1) You can mock the backend: make sure the right functions get called in 
response the user actions.
2) Simulate user actions by manually posting events, calling functions, etc.
3) Mock the gui: make sure the backend is calling all the right GUI 
functions.

Try to keep your backend and gui as independent as possible, it will 
make testing much easier.

> 
> Thank you.
> 
> 
> ------------------------------------------------------------------------
> 
> --
> http://mail.python.org/mailman/listinfo/python-list

I typically make a bunch of different suites that can be run 
individually, in various combinations, or all together. Testing I/O 
tends to be very slow, so it's nice to be able to turn of these tests 
when you're working on other parts of the system.

There are many other testing tools besides unittest. The one I use the 
most is coverage.py, to determine if my test suites are hitting all my 
code. But you can check out pymock, doctest, nose, etc. too. You may 
have to use a combination of tools to meet your needs.

There is also the Testing in Python mailing list 
(http://lists.idyll.org/listinfo/testing-in-python). You can probably 
get some advice there as well. The more specifics you can give, the better.

As you solve your testing problems, please take the time to post your 
solutions (if you can) so we can all learn from your experiences.

-Matt