Unit testing data; What to test

[Sean 'Shaleh' Perry]

On Sun, Feb 18, 2001 at 07:09:59AM +0000, John J. Lee wrote:
> 
> Having decided testing was a Good Thing and that I ought to do it, I've
> started to write tests, using PyUnit.
> 
> The second (third) one is vague: What / How should one test?  Discuss.
> 
> I'm sure there must be good and bad ways to test -- for example, I read
> somewhere (can't find it now) that you should aim to end up so that each
> bug generates, on (mode) average, one test failure, or at least a small
> number.  The justification for this was that lots of tests failing as a
> result of a single bug are difficult to deal with.  It seems to me that
> this goal is a) impossible to achieve and b) pointless, since if multiple
> test failures really are due to a single bug, they will all go away when
> you fix it, just as compile-time errors often do.  No?
>

How I commonly see it done (and how I do it):

common cases
edge cases -- perhaps -1 could be valid in some cases and not others
              or maybe you have a buffer which is only soo big
bad cases -- but -2 never is

You want to see if your code fails or gives expected output.  The other big
thing is every time you find a bug, you fix it and then add a test case that
triggers the bug.  So if you run the buggy version you get an error and the
new version is clean.  This way, if you accidently remove the fix or cause
a similar bug elsewhere, your tests catch it.