Hardware take on software testing.

[Peter Hansen]

Terence Way wrote:
> 
> Cleanroom, and much of the software development methodologies, insist
> that bugs get more expensive as time goes on, so it's cost-effective
> to spend time up-front to remove all bugs.  XP refutes that, and says
> bugs cost the same no matter what: it's only software, just change it
> when you find a bug.

I'm unsure that's the way XP looks at it.  XP does three things, that
I can see, in relation to this.  

One is it avoids the very expensive "big design up-front", along with 
the masses of unused documentation, the length design sessions, and 
so forth, and therefore reduces the cost of coding, where the bugs 
are actually inserted, to a more reasonable level (and, incidentally, 
reduces the cost of bugs at the same time).  

The second is that it puts a project into maintenance mode 
from the first week, and therefore flattens the supposedly
rising curve of bug-cost as seen in traditional projects to the
level seen during maintenance.  (Hmm... never heard it described
that way... maybe that's not quite true, but it seems valid at
the moment.)

The third, and more important, is that it tries hard to avoid 
inserting bugs in the first place, with the extensive unit and
acceptance tests, and test-driven development.  If you don't
put most of the bugs in in the first place, you don't really 
get to worrying about the cost of the few remaining ones (and,
as you point out, you just change it.)

> Despite this, I wonder if the two can be combined: XP with cleanroom
> testing, resulting in peer-reviewed well-specified software with a
> MTTF quality metric.  Hrmm.

XP certainly produces the first two (peer-reviewed and well-specified)
already.  There are not good metrics coming out of XP projects yet,
however, in the area of MTTF.  (Note MTTF calculations are a black
art in themselves, however, and very few projects using _any_ 
methodology produce useful MTTF numbers, AFAIK.)

-Peter