[Python-Dev] Proposed unittest changes

[Ron Adam]

Michael Foord wrote:
> Guido van Rossum wrote:
>> I'm worried that a mass renaming would do anything but inconvenience
>> users during the already stressful 2->3 transition.
>>
>> I'm more in favor of the original proposal of reducing the redundancy post-3.0.
>>
>>   
> 
> So nix the PEP-8'ifying until after 3.0.
> 
> So new methods should follow the current naming scheme (assertIn, 
> assertNotIn etc).
> 
>> If you're looking for useful features, Google has a set of extensions
>> to unittest.py that I find useful:
>>
>> - module-level setUp and tearDown
>>   
> So when a suite is made from a module the setUp should be called before 
> the first test and tearDown after the last. I can look at that.
> 
>> - routines for comparing large lists and strings that produce useful
>> output indicating exactly where the inputs differ.
>>
>> - assertLess etc.
>>   
> By etc I assume you mean:
> 
>     assertLessThan
>     assertGreaterThan
>     assertLessThanOrEquals
>     assertGreaterThanOrEquals
> 
> Would not variants be useful as well - it seems not as the not of one is 
> always another... (I think 'assertLessThan' reads better than 
> 'assertLess' but will do what I'm told...)
> 
>> - assertSameElements (sort of like assert(set(x) == set(y))
>>   
> 
> Sounds good.
> 
> Did you look at the other proposals?
> 
> * Decorator to make a function a TestCase
> * Convenience RunTests functions taking modules, suites and TestCases 
> and running them
> * Improved messages for assertEquals and assertNotEquals when an 
> explicit message is passed in
> * Improved message when comparing lists/tuples with assertEquals
> * The additional asserts that I suggested (In/NotIn, RaisesWithMessage, 
> Is/NotIs)
> 
> I think that there is still work I can do on the docs even before any 
> grand renaming...


Something I found useful to improve messages is to have the tests catch and 
then report errors with it's own unique exceptions to indicate the failures.

What that does is make a very clear distinction between an error found in a 
module being tested vs test case.  An error in the test case results in a 
regular python exception and a trace back of the test case.  An error in 
the module being tested results in an specific unittest exception 
indicating what type of error was found in the module followed by the 
original exception and traceback from the module being tested.

Ron



class TestCase(unittest.TestCase):
     failureException = unittest.TestCase.failureException

     class Unexpected_Exception_Raised(failureException):
         def __init__(self, exc, ref):
             self.exc = exc
             self.ref = ref
         def __str__(self):
             return '\n'.join([repr(self.exc), '\nReference:', self.ref])

     class Wrong_Exception_Raised(Unexpected_Exception_Raised): pass

     class No_Exception_Raised(failureException):
         def __init__(self, result, ref=""):
             self.result = repr(result)
             self.ref = ref
         def __str__(self):
             return "returned -> " + '\n'.join([self.result,
                     '\nReference:', self.ref])

     class Wrong_Result_Returned(No_Exception_Raised):
         def __str__(self):
             return '\n'.join([self.result, '\nReference:', self.ref])

     def assertTestReturns(self, test, expect, ref=""):
         try:
             result = test()
         except Exception, e:
             e0, e1, e2 = sys.exc_info()
             raise self.Unexpected_Exception_Raised, (e, ref), e2
         if result != expect:
             raise self.Wrong_Result_Returned(result, ref)

     def assertTestRaises(self, test, expect, ref=""):
         try:
             result = test()
         except Exception, e:
             if isinstance(e, expect):
                 return e
             else:
                 e0, e1, e2 = sys.exc_info()
                 raise self.Wrong_Exception_Raised, (e, ref), e2
         raise self.No_Exception_Raised(result, ref)