Correct code/unit testing (was Re: Efficient python programming...)

[Kragen Sitaker]

Peter Hansen <peter at engcorp.com> writes:
> Kragen Sitaker wrote:
> > Peter Hansen <peter at engcorp.com> writes:
> > I also agree that unit tests are very useful for reducing bugginess.
> > 
> > But I do not think that they are either necessary or sufficient for
> > writing correct code.  
> 
> Without tests, how much confidence can you have in the correctness
> of your code?  It's been said "you don't know it works if you 
> don't test it".

Tests are one way to increase your confidence in the correctness of
your code, but they are not the only way.

> > I think (and extensive experimentation has
> > shown) that careful reasoning and code reviewing is necessary and
> > sufficient to write correct code.
> 
> Whoa!  "extensive experimentation has shown"??

Yes.  (Well, I guess "correct code" is probably overstating the case.
"Code with many fewer bugs per line than most code in use" is probably
more accurate.)  Are you familiar with Cleanroom programming?

> I would suggest that what extensive experience (I change the word
> deliberately) has shown many of us is that careful reasoning and
> code reviewing, no matter how extensive, will *always* leave bugs in
> the code.  Have you never spent a large amount of time carefully
> designing and coding a wonder little program, and checked it over
> very carefully, even with a peer, and later found a bug in it?

Of course!  People are fallible, it's much easier to do a bad code
review than a good one.

> Maybe you're just a better programmer than I, and many others, but I
> would say this inevitable outcome is the source of the term "egoless
> programming" -- if you have a big ego, you'll be devastated by the
> repeated proof that you will never, ever get it right the first
> time, or even after lots of reviewing.

Yes, one of the beautiful things about programming is how it impresses
upon each of us how fallible we are.

For what it's worth, the term "egoless programming" was invented by
Gerald Weinberg for his book _The Psychology of Computer Programming_
to describe a programming style in which group reviews focus on
finding bugs in code.  It comes from Weinberg's observation that code
reviews work very poorly if the author is defensive about bugs, and
much better if the author is trying to find bugs too.

Of course, it's true of testing too; testing works best if you're
trying to find bugs.

> > It is certainly possible to err too far in the direction of believing
> > your code correct simply because it passes its test suite, and this is
> > a common beginner's error.  I think it is also possible to err too far
> > in the direction of writing and running tests instead of desk-checking
> > code.
> 
> I can only suggest that until you have tried test-first development,
> in which the code is written in very, very small steps only to make
> small failing tests pass, and discovered the possibilities of 
> emergent design, and other aspects of this approach, then we will not
> find much common ground in this discussion.

Test-first development, exactly as you describe it above, has been my
preferred development style since mid-2000, at least when working on
programs of more than 100 lines, especially with other people.  I've
written a lot of code that way.

I think now we have each committed the same arrogant gaffe in this
discussion: "It is only possible to disagree with me if you do not
know what I am talking about."  I was wrong, and so are you.

> I simply disagree with your last sentence above, and I believe
> beginners _never_ write unit tests anywhere that I've seen, so I
> disagree with the first one too. :)

Beginners almost always test their code, but rarely by writing a
formal test suite --- usually by hand.

I think inspection can find bugs that are very difficult to find by
testing, and vice versa.  That is the reason for my second statement;
a balance of careful reasoning and testing can produce better code
than the same amount of time spent on either alone.