Code correctness, and testing strategies

[Nicola Musatti]

On Jun 3, 12:35 am, ja... at cd.chalmers.se (Jacob Hallen) wrote:
> In article <mailman.1539.1211660262.12834.python-l... at python.org>,
>
> David  <wizza... at gmail.com> wrote:
[...]
> >That's why you have human testing & QA. Unit tests can help, but they
> >are a poor substitute. If the customer is happy with the first
> >version, you can improve it, fix bugs, and add more unit tests later.
>
> The most important aspect of usnit testing is actually that it makes the code testable.
> This may sound lik an oxymoron but it is actually a really important property. Testable
> code has to have a level of modularity as well as simplicity and clarity in its
> interfaces that you will not achieve in code that lacks automated unit tests.

I don't agree. To me the most important aspect of unit testing is that
it is automated. That is, the time you spend writing new tests you
reap each time you run your test suite.

The fact that writing automated tests has a beneficial effect on the
modularity of your code is a pleasant though hard to measure side
effect.

Automation should also be the most convincing argument for the OP. The
most evident liability of the approach he described is the need to re-
instrument his code all over again each time. Writing an equivalent
set of automated tests is likely to take him more, but the time taken
to write each test is time he doesn't need to spend anymore.

[...]
> Another aspect that you are raising is the use of human testing and QA. I agree that
> these are important, but every bug they discover is a failure of the developers and
> their tests. Our testers can sail through a testbed in 30 minutes if there are no bugs.
> Every single bug adds 30-60 minutes of testers time in order to pinpoint the bug
> and supply the developers with enough information to locate and fix it. Add some
> 10 minutes to testing time on the next testbed to verify that the bug is actually
> fixed. In my end of the world, tester time is not cheaper than developer time. It
> is also a scarcer resource than developer time.

Moreover, hand performed testing takes the same amount of time each
time it is performed and doesn't enjoy the incremental aspect of
automated testing.

Cheers,
Nicola Musatti