Code correctness, and testing strategies

[Ben Finney]

David <wizzardx at gmail.com> writes:

> What strategies do you use to ensure correctness of new code?

Behaviour Driven Development <URL:http://behaviour-driven.org/>.

* Start with an automated unit test suite that passes.

* Describe, as formally or informally as you like, but *not* in code,
  the new or altered behaviour that the code should exhibit.

* Assert some simple aspect of that behaviour in an automated unit
  test that asserts one single true/false statement.

  This forces you to be absolutely specific about the new behaviour
  you're about to implement, at exactly the point where you need that
  specification.

  It also forces you to think about the code from the perspective of
  an external user of the code, *before* implementing it. This leads
  over time to much better-designed APIs, without large up-front
  planning.

* Run the entire unit test suite; watch the new test fail with an
  explanation of why it failed.

  At this point you know your unit test suite will fail, with a good
  explanatory message, in any future failure of the code to implement
  the specified behaviour.

* Alter the application in the simplest possible way to make the
  entire unit test suite pass. Don't be clever, don't optimise
  anything (yet); be lazy. Do the simplest thing that could possibly
  work.

  This ensures you don't have any code that isn't necessary to
  implement your specified requirements. Since you designed an
  implemented a test that will use this code first, you should have a
  much better idea of the simplest code that will satisfy it, and
  won't have to go down so many wasteful dead-ends.

* Run the entire unit test suite. Fix any failing tests.

  The existing tests in the unit test suite will show any regressions
  you introduced. You might need to re-evaluate the relevance of old
  tests as you specify and implement new behaviour.

* Only once the entire unit test suite passes again, refactor any code
  that now exhibits redundancies or inelegancies.

  This doesn't permit you to change behaviour or API; "refactor" means
  changing the structure of the code without changing its behaviour.

* Run the entire unit test suite. Fix any failing tests.

  You have now designed, specified, implemented, and tested the new
  behaviour. Check your code into your VCS with an appropriate message
  saying what you implemented and why.

* Repeat from the top.

> Specifically, if you've just written 100 new lines of Python code,
> then:
> 
> 1) How do you test the new code?

Write the tests first, and make sure you aren't writing so much new
code between tests.

> 2) How do you ensure that the code will work correctly in the future?

Always run your complete automated unit test suite between changes to
the code. If possible, arrange for the unit test suite to be run
automatically every time you change your working copy of the code
and/or tests. That way you only have to look at the output when you
want to know what effect you just had on the pass/fail status.

Treat any bug as "desired change to the behaviour of the code", and
begin by implementing a new unit test that fails because of the bug.
Then your unit test suite will catch any regression of the bug.

-- 
 \         “In case you haven't noticed, [the USA] are now almost as |
  `\         feared and hated all over the world as the Nazis were.” |
_o__)                                           —Kurt Vonnegut, 2004 |
Ben Finney