[Python-Dev] Checking in a broken test was: Re: [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py

[Scott David Daniels]

James Y Knight wrote:
> On Jan 8, 2006, at 1:01 PM, Martin v. Löwis wrote:
> 
>> Fred L. Drake, Jr. wrote:
>>
>>> I like the way trial (from twisted) supports this.  The test  
>>> method is written
>>> normally, in whatever class makes sense.  Then the test is marked  
>>> with an
>>> attribute to say it isn't expected to pass yet.  When the code is  
>>> fixed and
>>> the test passes, you get that information in trial's output, and  
>>> can unmark
>>> the test.  This avoids having to refactor test classes just to  
>>> update the
>>> status of a test.
>>>
>> So how is the mark added? I would suggest
>>
>>    @xfail
>>    def test_foo(self):
>>      self.assertEquals(0,1)
> 
>    def test_foo(self):
>      self.assertEquals(0,1)
>    test_foo.todo = "Why this test fails description message."

Looks pretty good.  Here's some code to critique:
     import unittest

     class BrokenTest(unittest.TestCase.failureException):
         def __repr__(self):
              return '%s: %s: %s works now' % (
                      (self.__class__.__name__,) + self.args)

     def known_broken_XXX(reason):
         def wrapper(test_method):
             def replacement(*args, **kwargs):
                 try:
                     test_method(*args, **kwargs)
                 except unittest.TestCase.failureException:
                     pass
                 else:
                     raise BrokenTest(test_method.__name__, reason)
             return replacement
         wrapper.todo = reason
         return wrapper

So your use looks like:

     class SomeTests(unittest.TestCase):
         ...
         @known_broken_XXX('Someday the naive expect this to work')
         def test_distinct_ones(self):
             self.failIfEqual(1, 1.0)



-- Scott David Daniels
Scott.Daniels at Acm.Org