Unit testing - suitable for all development?

[Greg Ewing (using news.cis.dfn.de)]

Kylotan wrote:
> I don't have any complex functions that perform some important
> mutation of the object's state which can be verified as being correct
> at the end with minimal interaction with other objects. It just
> doesn't happen at the moment.

It sounds like you have a similar situation as I have
with Pyrex. The Pyrex compiler is made up of a large
number of parts which interact in very complex ways.
Each part on its own is fairly simple, and while it
*could* be tested on its own after a fashion, this
wouldn't really help much, because most of the
interesting things happen in the interactions between
the parts, rather than inside the parts themselves.

I haven't bothered doing unit testing in the conventional
sense with Pyrex. What I have instead is a large collection
of very small Pyrex files, each of which exercises some
feature of the Pyrex language. Running the test consists
of compiling the file and checking that the generated code
matches previously verified output.

In other words, my "unit tests" test units of functionality,
rather than units of code. I find them to be quite adequate --
anything that would be caught by a code-unit test will
also be caught by some functionality-unit test, and the
traceback usually makes it clear which piece of code is
at fault.

With regard to regression testing, I do find it to be
extremely important. It's the only thing that gives me
the confidence to add new features to Pyrex, or to
reorganise the implementation of existing features,
without fear of ending up with a hopelessly broken
mess. Which I'm *certain* would have happened to Pyrex
long before now if I hadn't had automatic testing.

Your system may be small and simple enough so far that
you haven't experienced any breakage problems, but things
may not stay that way.  So I urge you to make your tests
automatable if at all possible, even if you don't feel you
need it at the moment. Chances are you'll thank yourself for
it later.

In summary: I don't think it's necessary to be too dogmatic
about what exactly constitutes a "unit test". What I do think
is important: (a) Build up a test suite that exercises as much
of the system's functionality as possible. (b) Automate the
testing process as much as possible and run the tests
frequently.

-- 
Greg Ewing, Computer Science Dept,
University of Canterbury,	
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg