[Tutor] writing effective unittests

[Luke Thomas Mergner]

* Albert-Jan Roskam <fomcl at yahoo.com> wrote: 

> 
> 
> > Hi,
> 
> > 
> > I am trying to learn a bit of test-driven programming using unittests and 
> > nosetests. I am having trouble finding resources that explain how to write 
> > effective tests. I am not a programmer or a student, so I do not have access to 
> > collegues or classes. I'm hoping that someone knows of a good tutorial that 
> > I've missed. Based on the O'Reilly 2-Part introduction, I hope that 
> > learning to write tests will make my code better. 
> > 


I am on the digest version of the list, so I haven't gotten a copy of any replies. Apologies if two copies of my last email hit the list, I'm still learning to use mutt.

To partially answer my own question, let me tell you what I've learned in the last 48 hours. It is easy to learn that unittest is the standard testing module in Python > 2.6, that it is backported, that it is being improved in Python 3 with new tools. There are a few PyCon videos that discuss this evolution. It is also easy to learn that unittest is based off of prior library for Java (JUnit) and SmallTalk. It is fairly easy to find an introduction to the syntax of writing a test:

>class blargTest(unittest.TestCase)
>	def testFeature(self):
>		... do some stuff...
>		assertTrue('blarg', blarg)

And there are a few videos available that walk through how to put this together into a test suite. (I've included what I found below, so that this email will have some value to others.)

What am I missing? The biggest problem is that no one is explaining the rationale behind testing. The trivial examples compare integers: 2 == 2. At first glance this seems pointless. I had assumed that tests would attempt to confuse my functions and teach me how to write more robust code. But I *think* now that tests are a way to determine if new code has changed old behaviors. Testing 2 == 2 is trivial, but if the function starts returning 3 in a few months, it would be helpful to know right away. In general, though, I'm finding it difficult to conceptualize what my tests should be doing, testing, and telling me about my code.

The second problem is complexity. I'll paraphrase a well known saying about regex: "I wrote a unittest for my Python code, now I have two problems." Just to test my relatively simple code, I'm trying to do things like create temporary file structures, do some datetime parsing, etc. None of this is easy code for me to write at this stage. When a test fails, it could just as easily be a problem with the test code. (I'm unaccountably reminded about infinite regress here.) I *think* the answer must be that the benefit of writing tests on large, complex projects far outweighs the cost. But I've seen even small projects tout their test coverage.

I think what I need is a conceptual shift: how do python programmers use unittests? 

My next step is to look into buying some real books. 

Resources

Agile Testing Blog: http://agiletesting.blogspot.com/
Tool Taxonomy: http://wiki.python.org/moin/PythonTestingToolsTaxonomy
Doug Hellman's Blog: http://www.doughellmann.com/articles/pythonmagazine/completely-different/2008-01-testing-tools/
Unittest Docs: http://docs.python.org/3/library/unittest.html
Nose Docs https://nose.readthedocs.org/en/latest/testing.html
Fixture Docs http://www.farmdev.com/projects/fixture/index.html

PyCon 2010: http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2010-new-and-improved-coming-changes-to-unittest-the-standard-library-test-framework-52-3283307

PyCon 2011: http://blip.tv/pycon-us-videos-2009-2010-2011/pycon-2011-testing-with-mock-4899484

PyOhio July 2012. This looks perfect except it is missing about half the audio. https://www.youtube.com/watch?v=jTJHQ-zQMk4

Marakana Tech TV: https://www.youtube.com/watch?v=xdY7svOz6n4

At O'Reilly: http://onlamp.com/pub/a/python/2004/12/02/tdd_pyunit.html
 


- Luke Mergner