Probabilistic unit tests?

[Steven D'Aprano]

On Fri, 11 Jan 2013 16:26:20 +0000, Alister wrote:

> On Thu, 10 Jan 2013 17:59:05 -0800, Nick Mellor wrote:
> 
>> Hi,
>> 
>> I've got a unit test that will usually succeed but sometimes fails. An
>> occasional failure is expected and fine. It's failing all the time I
>> want to test for.
>> 
>> What I want to test is "on average, there are the same number of males
>> and females in a sample, give or take 2%."
[...]

> unit test are for testing your code, not checking if input data is in
> the correct range so unless you are writing a program intended to
> generate test data I don't see why unit test are appropriate in this
> case.

I don't believe Nick is using unittest to check input data. As I 
understand it, Nick has a program which generates random values. If his 
program works correctly, it should generate approximately equal numbers 
of "male" and "female" values. So he writes a unit test to check that the 
numbers are roughly equal.

This is an appropriate test, although as I already suggested earlier, 
unit tests are not well suited for non-deterministic testing.


-- 
Steven