Code correctness, and testing strategies

[Jacob Hallen]

In article <mailman.1539.1211660262.12834.python-list at python.org>,
David  <wizzardx at gmail.com> wrote:
>> Clients generally require *working* software. Unfortunately it is all too
>> easy to ship something broken because then you can claim you completed the
>> coding on time and any slippage gets lost in the next 5 years of
>> maintenance.
>
>That's why you have human testing & QA. Unit tests can help, but they
>are a poor substitute. If the customer is happy with the first
>version, you can improve it, fix bugs, and add more unit tests later.

The most important aspect of usnit testing is actually that it makes the code testable.
This may sound lik an oxymoron but it is actually a really important property. Testable
code has to have a level of modularity as well as simplicity and clarity in its
interfaces that you will not achieve in code that lacks automated unit tests.

You can easily convince yourself that this is true by adding complete coverage unit
tests to some code after you have written it. It's tough work and more often than
not, you need to refactor the code to make it testable.

Another aspect that you are raising is the use of human testing and QA. I agree that
these are important, but every bug they discover is a failure of the developers and
their tests. Our testers can sail through a testbed in 30 minutes if there are no bugs.
Every single bug adds 30-60 minutes of testers time in order to pinpoint the bug
and supply the developers with enough information to locate and fix it. Add some
10 minutes to testing time on the next testbed to verify that the bug is actually
fixed. In my end of the world, tester time is not cheaper than developer time. It
is also a scarcer resource than developer time.

The roundtrips between developers and testers also add to real required to develop
the product. There will always be delays introduced by having to produce the tester
with a testable system, wait for the tests to happen, have the report written up
and allocated to a developer etc. With TDD most of these bugs will be caught
in a no-delay loop at the developers desk.

This last fact has another important property that is easily overlooked. Once the
off-by-one errors and other trivial bugs don't clutter the minds of the testers
they will start thinking more clearly about the application and where the really
nasty bugs are found - the ones where your thinking went wrong but your implementation
is correct. if the testers are busy tracking trivial bugs, your customers will
find the nasty one. If you are lucky, they will tell you.

Jacob Hallén

--