[Python-Dev] Disable tests in unittest (issue3202)

[Justin Mazzola Paluska]

Hi,

I just reported issue3202 in the bugracker
(http://bugs.python.org/issue3202) with a patch that implements a way
to disable unittest.TestCases using a decorator.  Benjamin Peterson
suggested that I bring up the issue on python-dev.

The idea behind the patch is that it’s sometimes useful to disable
tests without removing them from the TestCase.  For example, a
co-worker and I have a module with a few tests that will fail for the
forseeable future because we haven’t had a chance to implement the
features the tests are testing.  The tracebacks for these “not
implemented yet” tests obscure real failures in other tests.

Normally, we’d just rename the failing methods from something starting
with “test” to something like “_test”, so unittest ignores them.
However, doing so removes all traces of the tests when you re-run the
test suite, so the disabled tests are easy to forget.

Instead, issue3202 allows you to write something like:

  from unittest import main, TestCase, disabled
  
  class SampleTest(TestCase):
  
      def test_addition(self):
  
          self.assertEqual(2, 1+1)
  
      def test_broken(self):

          # this is a real failure in our code      
          self.assertEqual(5, 2+2)

      @disabled
      def test_not_implemented(self):
  
          # Test of a feature not implemented yet.
          doit()
  if __name__ == '__main__':
      main()

which has one test disabled.  Running the test suite outputs

    %python sample_tests2.py 
    .FD
    ======================================================================
    FAIL: test_broken (__main__.SampleTest)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "sample_tests2.py", line 12, in test_broken
        self.assertEqual(5, 2+2)
    AssertionError: 5 != 4
    
    ----------------------------------------------------------------------
    Ran 3 tests in 0.001s
    
    FAILED (failures=1)

showing only the real failure and a simple “D” for the disabled test.
The disabled test is still there, but doesn’t get in your way.

JUnit4 has a similar decorator for its tests, @Ignore.

The patch in issue3202 implements the decorator @disabled and a few
modifications to the classes in the unittest module.

What does the Python hivemind think?  I’m also open to enhancing it if
the list has any ideas.
	—Justin