[SciPy-Dev] User Acceptance Testing

[Pauli Virtanen]

On Fri, 13 May 2011 19:40:44 -0400, Joshua Grant wrote:
> Hi everyone. So I notice that for Scipy, there's some fairly extensive
> unit tests, which are great for the actual development of the library
> code. Are there any forms of user acceptance testing, automated or
> otherwise? Ways to simulate user behaviour and larger blocks of code,
> beyond single methods/functions?

You can write a given functional test in the form of a unit test -- 
just solve a problem as you would do normally, and put asserts at
the end to check that the result is correct. In fact, this is what
a majority of the unit tests in Scipy do; they're (fixed) functional
tests for high-level routines.

I'm not sure how you would generate tests automatically for library
code (do you have some ideas :)? With UIs and web pages I can understand
that one can do fuzz testing to simulate user input, and check that
whatever is done does not cause totally unexpected results.  But when
the user input is scientific code, the specifications are so complex that
I don't see a way to generate that automatically --- it's easier to write
out given cases manually.

-- 
Pauli Virtanen