PyUnit and multiple test scripts

[Peter Hansen]

Calvin Spealman wrote:
> I'm trying to find the best way to use PyUnit and organize my test scripts.
> What I really want is to separate all my tests into 'test' directories
> within each module of my project. 

The script below will do that.

> I want all the files there to define a
> 'suite' callable and to then all all those suites from all those test
> directories into one big suite and run it all. I'm having trouble with
> this.

It won't do that.  In my own opinion, based on experience, this isn't
even a desirable thing for several reasons.  For one thing, using that
whole "suite" thing seems to be a lot of work for little gain.  The
default "unittest" stuff will already find all classes that are instances
of unittest.TestCase, and will already run all methods in them that
begin with the string "test".  Furthermore, it's often much better to
run different test files using separate processes, mainly to ensure
you don't pollute one test's starting conditions by failing to clean
up stuff (e.g. terminate all threads) from a previous test.

> 1) Is there a very simple way to just take a file path and name, that I
> could use to open the source file, and load it as a module object, no
> strings attached?
> 
> 2) Is there already a framework around that will do what I need?

Try this.  It works here (in a more complex version: this was pruned
for posting here).  Basic conditions: all tests are in subfolders
called "test", and all have a filename ending with _unit.py.  (This
lets me name them based on the module that they are testing.)  You can
change that pattern in the code.  It's possible there are external
dependencies on things unique to my environment, but I've tried
to catch and remove them all.

Note (though it's unrelated to this particular script) that since the
tests run in a subfolder of the folder where the code under test resides,
you need to do the equivalent of "sys.path.append('..')" in each
test file.  I accomplish that by having all import my own library
module called "testbed", which does this during import time:

   srcDir = os.path.abspath('..')
   sys.path.insert(1, srcDir)

Anyway, it might give you some ideas.

'''test runner'''

import os
import sys
import subprocess
import fnmatch


def runTest(path):
     folder, filename = os.path.split(path)
     cmd = [sys.executable, '-u']

     p = subprocess.Popen(cmd + [filename],
         cwd=folder or '.',
         stdout=subprocess.PIPE, stderr=subprocess.STDOUT,
         universal_newlines=True)
     output = []
     line = []
     while True:
         c = p.stdout.read(1)
         if not c:
             break
         if not line:
             sys.stdout.write('--|')
         sys.stdout.write(c)
         line.append(c)
         if c == '\n':
             output.append(''.join(line))
             line = []

     return output


def findfiles(path, pattern):
     '''scan all files and folders below path, returning sorted list of those 
matching pattern'''
     files = []
     match = fnmatch.fnmatch
     for p, ds, fs in os.walk(path):
         for f in fs:
             path = os.path.abspath(os.path.join(p, f))
             if match(path, pattern):
                 print path
                 files.append(path)

     files.sort()
     return files


def run():
     pattern='*/tests/*_unit.py'
     files = findfiles('.', pattern)
     if not files:
         print 'no tests found'
         sys.exit(1)

     print 'running all tests below %s' % os.path.abspath('.')

     try:
         for file in files:
             if not os.path.exists(file):
                 print 'file not found', file
                 continue
             print
             print file
             output = runTest(file)


     except KeyboardInterrupt:
         print 'User aborted test.'


if __name__ == '__main__':
     run()


# EOF