Code correctness, and testing strategies

[David]

> I work in small increments, writing one test at a time, then some code,
> then another test, then some more code, etc.  In fact, I take this to what
> many people might call an extreme.

Thanks for the replies.

I've read about the TDD (and similar) approaches. Maybe I need to try
it and get used to it, but there are a few things I don't like about
it on a gut level. I'll try to enumerate my concerns here.

Problem 1: You can only code against tests

Basically, with TDD you write the tests first, then the code which
passes/fails the tests as appropriate. However, as you're writing the
code you will also think of a lot of corner cases you should also
handle. The natural way to do this is to add them to the code first.
But with TDD you have to first write a test for the corner case, even
if setting up test code for it is very complicated. So, you have these
options:

- Take as much time as needed to put a complicated test case in place.
- Don't add corner case to your code because you can't (don't have
time to) write a test for it.
- Add the corner case handling to the code first, and try to add a
test later if you have time for it.

Problem 2: Slows down prototyping

In order to get a new system working, it's nice to be able to throw
together a set of modules quickly, and if that doesn't work, scrap it
and try something else. There's a rule (forget where) that your first
system will always be a prototype, regardless of intent.

With TDD, you have to first write the tests for the first version.
Then when that first version doesn't work out, you end up scrapping
the tests and the code. The time spent writing the tests was wasted.

Problem 3: Slows down development in general

Having to write tests for all code takes time. Instead of eg: 10 hours
coding and say 1/2 an hour manual testing, you spend eg: 2-3 hours
writing all the tests, and 10 on the code.

Problem 4: Can make refactoring difficult.

If you have very complete & detailed tests for your project, but one
day you need to change the logic fundamentally (maybe change it from
single-threaded to multi-threaded, or from running on 1 server to
distributed), then you need to do a large amount of test refactoring
also. The more tests you have (usually a good thing), the longer it
will take to update all the tests, write new ones, etc. It's worse if
you have to do all this first before you can start updating the code.

Problem 5: Tests are more important than code

You need to justify the extra time spent on writing test code. Tests
are nice, and good to have for code maintainability, but they aren't
an essential feature (unless you're writing critical software for life
support, etc). Clients, deadlines, etc require actual software, not
tests for software (that couldn't be completed on time because you
spent too much time writing tests first ;-)).

Problems like the above sum up why I'm not comfortable with the idea of TDD.

I think that automated tests can be very valuable for maintainability,
making sure that you or other devs don't break something down the
line. But these benefits must be worth the time (and general
inconvenience) spent on adding/maintaining the tests.

If I did start doing some kind of TDD, it would be more of the 'smoke
test' variety. Call all of the functions with various parameters, test
some common scenarios, all the 'low hanging fruit'. But don't spend a
lot of time trying to test all possible scenarios and corner cases,
100% coverage, etc, unless I have enough time for it.

I'm going to read more on the subject (thanks to Ben for the link).
Maybe I have some misconceptions.

David.