Hardware take on software testing.

[Terence Way]

On Fri, 06 Jun 2003 09:50:41 -0400, Paddy McCarthy wrote:

> If software were tested like hardware...
> 
What you're describing sounds a lot like IBM's Cleanroom: which as I
recall had two broad principles:
1. Specify everything and hammer out flaws in review meetings; and
2. Rigorously track bugs (counted from the first time a file is
compiled!), using random test data that matches the expected usage.

If followed rigorously, then your S curve showed up, and the MTTF
(Mean Time To Failure) could be calculated for a piece of software.

I've used Cleanroom, and I have to honestly say I didn't have the
discipline to carry it through -- which my team picked up on.  The
project we worked on didn't fail, but in the end bugs were found and
stamped out the old-fashioned way.

Cleanroom, and much of the software development methodologies, insist
that bugs get more expensive as time goes on, so it's cost-effective
to spend time up-front to remove all bugs.  XP refutes that, and says
bugs cost the same no matter what: it's only software, just change it
when you find a bug.

Despite this, I wonder if the two can be combined: XP with cleanroom
testing, resulting in peer-reviewed well-specified software with a
MTTF quality metric.  Hrmm.