Unit-testing single function with large number of different inputs

[Robert Brewer]

Edvard Majakari wrote:
> Code I'm working on has to parse large and complex files and detect
> equally complex and large amount of errors before the 
> contents of the file is fed to module interpreting it.
> [snip]
> Thus I came up with the simple idea of adding two directories:
> test/correct-files and test/incorrect-files. Every time the 
> test suite is
> run, it goes through every file in the test/correct-files and
> test/incorrect-files and expects to receive status 0 from all 
> files under
> correct-files, and appropriate error code from all files under
> test/incorrect-files. How does it deduce the correct error code? Well,
> I've forced things to that any file in the test/incorrect-files must
> begin with code prefix 'xx-' in the file name, and the test 
> suite reads
> expected error code from the file name prefix. 

Good idea!

> The system has but one annoyance: first, if I use 
> self.assertEqual in the
> test_load method, it aborts on the first error found and as 
> such doesn't
> test all the other files.
> Any suggestions?

You *are* familiar with assertRaises, aren't you?

>From the 2.3 docs:

assertRaises( exception, callable, ...)
failUnlessRaises( exception, callable, ...)

Test that an exception is raised when callable is called with any
positional or keyword arguments that are also passed to assertRaises().
The test passes if exception is raised, is an error if another exception
is raised, or fails if no exception is raised. To catch any of a group
of exceptions, a tuple containing the exception classes may be passed as
exception.


HTH!

Robert Brewer
MIS
Amor Ministries
fumanchu at amor.org