A better unittest

[Thomas Heller]

I'm moving more and more to TDD (test driven development [1]), which
makes tests an integral part of the development process.  Failing tests
are ocurring very often with this development style, because you write
the tests first and then implement the functionality.

Having said this, I'm not really happy with the output of
unittest. Consider the following testcase:

#-----
class FailingTests(unittest.TestCase):
    def test_exception(self):
        return 1/0

    def test_fail(self):
        self.fail("fail on purpose")

    def test_failIf(self):
        self.failIf(1, "failIf on purpose")

if __name__ == '__main__':
    unittest.main()
#-----

Running this code, you get this output:

-----start output-----
EFF
======================================================================
ERROR: test_exception (__main__.FailingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\tss5\components\_Pythonlib\xunit\xunit.py", line 32, in test_exception
    return 1/0
ZeroDivisionError: integer division or modulo by zero

======================================================================
FAIL: test_fail (__main__.FailingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\tss5\components\_Pythonlib\xunit\xunit.py", line 35, in test_fail
    self.fail("fail on purpose")
  File "C:\Python22\lib\unittest.py", line 254, in fail
    raise self.failureException, msg
AssertionError: fail on purpose

======================================================================
FAIL: test_failIf (__main__.FailingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\tss5\components\_Pythonlib\xunit\xunit.py", line 38, in test_failIf
    self.failIf(1, "failIf on purpose")
  File "C:\Python22\lib\unittest.py", line 258, in failIf
    if expr: raise self.failureException, msg
AssertionError: failIf on purpose

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (failures=2, errors=1)
-----end output-----

It can be seen that the codeline in the unittest module where the
failure is raised is included in the output.

It has been discussed before that raising an exception in the callers
frame is not currently possible, but it is possible to change the
unittest output to not print the innermost stack frame.

I've patched unittest to do this, and now the output looks like this:

-----start output-----
EFF
======================================================================
ERROR: test_exception (__main__.FailingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\tss5\components\_Pythonlib\xunit\xunit.py", line 32, in test_exception
    return 1/0
ZeroDivisionError: integer division or modulo by zero

======================================================================
FAIL: test_fail (__main__.FailingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\tss5\components\_Pythonlib\xunit\xunit.py", line 35, in test_fail
    self.fail("fail on purpose")
AssertionError: fail on purpose

======================================================================
FAIL: test_failIf (__main__.FailingTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "C:\tss5\components\_Pythonlib\xunit\xunit.py", line 38, in test_failIf
    self.failIf(1, "failIf on purpose")
AssertionError: failIf on purpose

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (failures=2, errors=1)
-----end output-----

The big advantage is that if I execute the tests from within XEmacs with
C-c C-c, the output ocurrs in a *Python Output* buffer, and the cursor
in the source buffer jumps to the real cause of the error: the
self.failxxx() call.

What do people think? Is this a useful change, and should I submit a
patch?

Thomas

[1] Kent Beck: Test Driven Development: by example. Addison Wesley, 2002.
    ISBN 0-321-14652-0