When is unit-testing bad? [was: Re: does lack of type...]

[John Roth]

"John J. Lee" <jjl at pobox.com> wrote in message
news:878ys06tcv.fsf_-_ at pobox.com...
> Donn Cave <donn at u.washington.edu> writes:
>
> > In article <3EEE7D4C.DA0B3512 at engcorp.com>,
> [...]
> > He's not the only one who gets a message something like that
> > from the unit testing advocates on c.l.p.  Without the hyperbolic
> > ``all possible situations'', sure, but more to the point, does
> > the discussion usually acknowledge significant exceptions, or
> > rather tend to dismiss them?
>
> Nobody can rationally disagree with you that there might exist
> situations where writing tests is inefficient (saves less time than it
> costs).  But, of course, we'd like to know *what those cases are*,
> because, AFAIK, they haven't been clearly pointed out.
>
> I assume you have no problem accepting that many people have found
> that writing lots of automated unit tests, and writing them early,
> works better than not doing so.
>
> So, what criteria do you have for the test/no-test decision that work
> better than my own rule of "always write an automated unit test
> (probably before writing the code), unless I'm too lazy or
> incompetent" <0.2 wink>? (lazy in the bad sense, that is)
>
> I suppose people might argue that if the answers to the following
> questions, for example (any others to suggest?), are mostly 'no', then
> the ratio of tests to code should be lower:
>
> 1. Code likely to be reused in future?
> 2. Code has been reused in past?
> 3. Code will not grow big (say, > 1000 lines)?
> 4. Lots of people working on it?
> 5. For a long time?
> 6. Correctness of code particularly important?
> 7. Emphasis on refactoring?
>
>
> But you're probably already expecting the answers:
>
> 1. Often hard to tell if it will be.
> 2. But adding tests later is more expensive (as is fixing bugs later).
> 3. See 1, 2.
> 4. See 1, 2.  But even if it's only one person, it's usually worth it
(IME).
> 5. See 1, 2.
> 6. I have to admit code correctness is *not* always crucial (whatever
>    Bertrand Meyer may try to tell you), but I suspect that, usually,
if
>    it's important enough to fix bugs, it's worth writing tests.
> 7. My mind isn't entirely made up, but it might be suggested that if
>    you're not refactoring, you've made another mistake.
>
>
> Looking at the thread from which I'm trying to start this one, a
> lesson to be learned from the static vs. dynamic typing thing is that
> it doesn't always work to consider one thing at a time.  Static-typing
> wonks like to point out that static analysis finds bugs.  But us
> dynamic typing, um, types, note that static analysis isn't the only
> variable: dynamic typing brings other pros, static typing other cons,
> and (dynamic typing + tests) is better than the other combinations.
> So maybe unit-testing enthusiasts should be thinking more about the
> costs and opportunity costs of testing?  Maybe if we all switched to
> constraint-based / functional languages with static type inference and
> god-knows-what-else, and spent some of our testing time doing more
> code review, we'd be better off still?

You make an excellent point: it's the combination of benefits and
costs that make or break a programming language, as well as the
environment in which it's typically used.

Some day, someone will build an inference engine for formal
verification that works as fast as the tests in TDD, and then I
expect we'll see another shift.

John Roth


>
>
> John