unittest: new reporting category "skipped"

[Roy Smith]

Remy Blank <remy.blank_asps at pobox.com> wrote:
> In a project I am working on, I have a series of tests that have
> to be run as root, and others as a normal user.

Having just completed a project which required testing as root, I can 
sympathize with you completely!  If at all possible, try to avoid 
getting painted into that particular corner.

Is it possible that by building the appropriate test scaffolding, you 
could run the tests as a normal user?  For example, let's say the reason 
you need to be root is because you are reading a file which is mode 600.  
You could then do something like this:

class myFile:
   """Stub replacement for file class.  Flesh this out enough to
   supply whatever functionality your unit tests require."""

   def __init__ (self, filename):
      self.data = """this is line one
                     this is line two
                     this is line three"""

   def read (self):
      for line in self.data:
         yield line

def myFileFactory (filename):
   return myFile (filename)

file = myFileFactory

It's not always possible or practical to do that, but it's worth 
thinking about.

Assuming this isn't going to work for you, then I do like your idea of 
having a way to skip tests.  Sounds like you want to subclass 
TestResult, and add a addSkipped() method.  You'd then probably need to 
also subclass the appropriate TestRunner class to report the skipped 
tests.

Another way would be to subclass TestSuite and override addTest(), but 
my fear here is that skipped tests would just get quietly dropped.  You 
really want them to be tracked and alerted in the final report.  You're 
done testing when you get no failures and no skipped tests.  Unless you 
have a way to report the skips up to the top level TestRunner, it's too 
easy to lose track of the skips.

Your idea of signaling skips with exceptions seems reasonable.  You 
might want to have more than one kind:

class SkippedTest
class NotRoot (SkippedTest)
class MissingTestResource (SkippedTest)
class TakesTooLong (SkippedTest)

Of course, the unit test gurus would probably be horrified at the whole 
idea of skipping tests.  A test either passes or fails, and you run 
every test, every time.  Turn all the dials up to ten, and all that.  
It's your project, you decide what makes sense.