[Doc-SIG] Test set!

Edward D. Loper edloper@gradient.cis.upenn.edu
Wed, 14 Mar 2001 00:33:37 EST


I've finished the first version of my StructuredText test suite module.
Excerpted from the module's documentation:

  This module defines an organized, comprehensive set of test cases
  for STminus, STNG, and STpy.  Each test case consists of a
  self-documenting StructuredText string.  The test cases are
  hierarchically organized into 'TestSets'.

To just look at what tests are currently contained in the test set,
look at STminus002's test results, at:

  http://www.cis.upenn.edu/~edloper/pydoc/stminus002-test.html
  
It may seem like I've included too many test cases, but I've 
surprised myself at how often test cases will unexpectedly
stop working when I make minor changes to STminus.

Any suggestions for additional test cases are most welcome.  I'll
be adding test cases as time goes by.

If you want to actually use the test set, download its module from:

  http://www.cis.upenn.edu/~edloper/pydoc/sttest.py

The module defines 2 useful variables: 
   everything -- a 'TestSet' containing all tests.
   test_hierarchy -- a hierarchy containing all tests.

The easiest way to use it is to evaluate::

   everything.tests('+STNG')

or::

   everything.tests('+STpy')

Which will return a list of strings, containing the test cases for
each variant (currently these will return the same thing).

If you want to do regression testing, you should:
  1. run the test suite, output to a format of your choice, and
     check all the answers by hand.
  2. save said output to a file
  3. to do regression testing after you change your implementation,
     run the test suite again, and diff it against your saved
     "known-correct" results.

-Edward