Code correctness, and testing strategies

[Jacob Hallen]

In article <mailman.1532.1211644302.12834.python-list at python.org>,
David  <wizzardx at gmail.com> wrote:
>> I work in small increments, writing one test at a time, then some code,
>> then another test, then some more code, etc.  In fact, I take this to what
>> many people might call an extreme.
>
>Thanks for the replies.
>
>I've read about the TDD (and similar) approaches. Maybe I need to try
>it and get used to it, but there are a few things I don't like about
>it on a gut level. I'll try to enumerate my concerns here.
>
>Problem 1: You can only code against tests
>
>Basically, with TDD you write the tests first, then the code which
>passes/fails the tests as appropriate. However, as you're writing the
>code you will also think of a lot of corner cases you should also
>handle. The natural way to do this is to add them to the code first.
>But with TDD you have to first write a test for the corner case, even
>if setting up test code for it is very complicated. So, you have these
>options:
>
>- Take as much time as needed to put a complicated test case in place.
>- Don't add corner case to your code because you can't (don't have
>time to) write a test for it.
>- Add the corner case handling to the code first, and try to add a
>test later if you have time for it.

As you come up with the corner case, write a test for it and leave the implementation
for later. The hard part of coding is always defining your problem. Once it is
defined (by a test) the solution is just a matter of tidy work.

>Problem 2: Slows down prototyping
>
>In order to get a new system working, it's nice to be able to throw
>together a set of modules quickly, and if that doesn't work, scrap it
>and try something else. There's a rule (forget where) that your first
>system will always be a prototype, regardless of intent.
>
>With TDD, you have to first write the tests for the first version.
>Then when that first version doesn't work out, you end up scrapping
>the tests and the code. The time spent writing the tests was wasted.
>

Agreed. There is no good way of reusing your prototype code in TDD. You end
up having to throw your propotype away in order to have a proper tested
implementation in the end. Takes more time up front, but less time over the
lifecycle of the program you are building.

>Problem 3: Slows down development in general
>
>Having to write tests for all code takes time. Instead of eg: 10 hours
>coding and say 1/2 an hour manual testing, you spend eg: 2-3 hours
>writing all the tests, and 10 on the code.

This is incorrect. It speeds up development in general. The debugging phase of
development becomes much shorter because the bugs are fewer and the ones you
have a much shallower. There are so many tests that reduce the scope in which
you have to search for the bug that it usually becomes trivial to find.

I have direct experience from this, getting my company to change to TDD about
10 months ago. Productivity has improved enormously. I'd say that we have cut
between 25 and 50% in development time.

>Problem 4: Can make refactoring difficult.
>
>If you have very complete & detailed tests for your project, but one
>day you need to change the logic fundamentally (maybe change it from
>single-threaded to multi-threaded, or from running on 1 server to
>distributed), then you need to do a large amount of test refactoring
>also. The more tests you have (usually a good thing), the longer it
>will take to update all the tests, write new ones, etc. It's worse if
>you have to do all this first before you can start updating the code.

No, this is a total misunderstanding. It makes refactoring much easier.
It takes a bit of time to refactor the affected tests for the module, but you
gain so much by having tests that show that your refactoring is not breaking
code that should be unaffected that it saves the extra time spent many times
over.

Introducing bugs because you missed some aspect in a refactoring is one of the
most common problems in non-TDD code and it is a really nasty quality
concern.
>
>Problem 5: Tests are more important than code
>
>You need to justify the extra time spent on writing test code. Tests
>are nice, and good to have for code maintainability, but they aren't
>an essential feature (unless you're writing critical software for life
>support, etc). Clients, deadlines, etc require actual software, not
>tests for software (that couldn't be completed on time because you
>spent too much time writing tests first ;-)).
>
The tests are as important as the code. As a customer, I don't think I'd buy
software today unless I know that it has been built using TDD. Certainly I am
ahead of the curve in this, but it won't be long before this will be required
by skilled organisations buying software and sooner or later the rest of the
world will follow.

Getting into a TDD mindset is hard work, but those who succeed produce better
software with less effort.

Jacob Hallén

--