[Tutor] writing effective unittests

[Brian van den Broek]

On 3 January 2013 14:46, Luke Thomas Mergner <lmergner at gmail.com> wrote:
> * Albert-Jan Roskam <fomcl at yahoo.com> wrote:
>
>>
>>
>> > Hi,
>>
>> >
>> > I am trying to learn a bit of test-driven programming using unittests and
>> > nosetests. I am having trouble finding resources that explain how to write
>> > effective tests. I am not a programmer or a student, so I do not have access to

<snip>

> To partially answer my own question, let me tell you what I've learned in the last 48 hours. It is easy to learn that unittest is the standard testing module in Python > 2.6, that it is backported, that it is being improved in Python 3 with new tools. There are a few PyCon videos that discuss this evolution. It is also easy to learn that unittest is based off of prior library for Java (JUnit) and SmallTalk. It is fairly easy to find an introduction to the syntax of writing a test:

<snip>

> And there are a few videos available that walk through how to put this together into a test suite. (I've included what I found below, so that this email will have some value to others.)
>
> What am I missing? The biggest problem is that no one is explaining the rationale behind testing. The trivial examples compare integers: 2 == 2. At first glance this seems pointless. I had assumed that tests would attempt to confuse my functions and teach me how to write more robust code. But I *think* now that tests are a way to determine if new code has changed old behaviors. Testing 2 == 2 is trivial, but if the function starts returning 3 in a few months, it would be helpful to know right away. In general, though, I'm finding it difficult to conceptualize what my tests should be doing, testing, and telling me about my code.


Hi Luke,

First, I should like to commend your post; collecting together the
results of your research and posting it is a valuable thing to do.
While I agree with Steven that you'd be better off not taking the list
as a digest, for me at least, the mild irritation that digest use
imposes is small compared to the goodwill the effort shown produces.

Second, I don't test nearly as much as I ought. So, read with a grain
of salt :-)

Now, for the meat.

A key idea that was implicit in Steven's first reply might be better
made explicit. When testing, you want to look for odd or edge cases
where things might break and embed tests that will let you know if
they do.

What counts as an edge or odd case is obviously context dependent.
But, if dealing with an arithmetical function of one input, you'd
generally want tests for at least 0, 1, -1, some large positive, some
large negative, and small and large positive and negative non-integer
rational numbers. If writing against 2.x, you'd also want an integer
large enough to be a long rather than an int, etc.  If testing a
function that takes one string, you'd want tests for at least the
empty string, a string consisting solely of whitespace, a single
character string, a string with "odd   whitespacing   that you
wouldn  '  t expect", strings in mixed case, ones with punctuation,
etc.

Another principle that I use to generate tests: while writing code,
every time I find myself considering some possibility and thinking
"But, that's impossible!" I try to put in a test to ensure that the
impossible actually is. http://www.youtube.com/watch%3Fv%3DD58LpHBnvsI

I didn't try to find the context, but testing 2 == 2 does seem a bit
pointless. However, a similar sort of test might be worthwhile.
Consider:

>>> class A(object):
	def __eq__(self, other):
		return False

	
>>> a=A()
>>> a==a
False
>>>

Now, one might think: "But that's such a stupid bug. I'd never do
that". If so tempted, go watch the link above :-)

In seriousness, the way that sort of test will help isn't so much to
confirm that the code works as intended when written, but that it
continues to work several moths later when you've forgotten most of it
and need to tweak just one thing that couldn't possibly affect other
parts of the code.

HTH,

Brian vdB