Hardware take on software testing.

[Steven Taschuk]

Quoth Michele Simionato:
  [...]
> somebody came up with many very subtle corner cases where the recipe 
> couldn't work. Not real bugs but limitations or warts. Some of them I
> knew, other where inexpected. Now my point is that TDD gave me a 
> sense of false security (similar to what you get when your program 
> compile in a compiled language) and made me too much confident.

Fwiw, I agree entirely with your point that sometimes (rarely) TDD
does not provide adequate coverage of cases.  What agile
methodologies gain you in this case, imho, is practical experience
with candidate algorithms and direct experience of the subtlety of
the subject.  This is very valuable.

Now that you have that experience, you might be motivated to throw
away what you have and take a more formal approach: Formalization
of the Python type system, formal characterization of the
circumstances under which the metaclass/metatype conflict occurs,
proof that such-and-such an algorithm avoids those circumstances,
you know the drill.

Imho such methods are worth it in such cases.  (Type system
hacking is one area where I *do* want proofs of correctness.)  But
an important point: you would not know for certain that this was
needed if you hadn't used agile methodologies to explore the
domain in the first place.

  [...]
> In the same sense that one should be skeptical about the confidence he gets 
> when the program compile, he should be skeptical about the confidence he gets 
> when the program passes the test suite.

In the same sense, yes.  But to a much lesser degree, of course;
the test suite is smarter than the compiler.

-- 
Steven Taschuk             "The world will end if you get this wrong."
staschuk at telusplanet.net     -- "Typesetting Mathematics -- User's Guide",
                                 Brian Kernighan and Lorrinda Cherry