[Python-checkins] CVS: python/dist/src/Lib/test README,1.7,1.8 test_difflib.py,1.2,1.3 test_doctest.py,1.1,1.2
Tim Peters
tim_one@users.sourceforge.net
Wed, 23 May 2001 00:46:38 -0700
Update of /cvsroot/python/python/dist/src/Lib/test
In directory usw-pr-cvs1:/tmp/cvs-serv5835/python/dist/src/Lib/test
Modified Files:
README test_difflib.py test_doctest.py
Log Message:
Remove test_doctest's expected-output file.
Change test_doctest and test_difflib to pass regrtest's notion of
verbosity on to doctest.
Add explanation for a dozen "new" things to test/README.
Index: README
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/README,v
retrieving revision 1.7
retrieving revision 1.8
diff -C2 -r1.7 -r1.8
*** README 2001/05/23 04:57:49 1.7
--- README 2001/05/23 07:46:36 1.8
***************
*** 1,6 ****
! Writing Python Regression Tests
! -------------------------------
! Skip Montanaro
! (skip@mojam.com)
--- 1,6 ----
! Writing Python Regression Tests
! -------------------------------
! Skip Montanaro
! (skip@mojam.com)
***************
*** 27,30 ****
--- 27,31 ----
your regression test cases.
+
PyUnit based tests
***************
*** 43,46 ****
--- 44,48 ----
lower-case names with words separated with underscores.
+
doctest based tests
***************
*** 49,62 ****
interactive session, including the Python prompts, statements that would
be typed by the user, and the output of those statements (including
! tracebacks!). The module in the test package is simply a wrapper that
! causes doctest to run over the tests in the module. The test for the
! doctest module provides a convenient example:
! import doctest
! doctest.testmod(doctest, verbose=1)
See the documentation for the doctest module for information on
writing tests using the doctest framework.
"traditional" Python test modules
--- 51,72 ----
interactive session, including the Python prompts, statements that would
be typed by the user, and the output of those statements (including
! tracebacks, although only the exception msg needs to be retained then).
! The module in the test package is simply a wrapper that causes doctest
! to run over the tests in the module. The test for the difflib module
! provides a convenient example:
!
! from test_support import verbose
! import doctest, difflib
! doctest.testmod(difflib, verbose=verbose)
! If the test is successful, nothing is written to stdout (so you should not
! create a corresponding output/test_difflib file), but running regrtest
! with -v will give a detailed report, the same as if passing -v to doctest
! (that's what importing verbose from test_support accomplishes).
See the documentation for the doctest module for information on
writing tests using the doctest framework.
+
"traditional" Python test modules
***************
*** 72,91 ****
If you are writing test cases for module spam, you need to create a file
! in .../Lib/test named test_spam.py and an expected output file in
! .../Lib/test/output named test_spam ("..." represents the top-level
! directory in the Python source tree, the directory containing the configure
! script). From the top-level directory, generate the initial version of the
! test output file by executing:
./python Lib/test/regrtest.py -g test_spam.py
! (If your test does not generate any output when run successfully, this
! step may be skipped; no file containing expected output will be needed
! in this case.)
Any time you modify test_spam.py you need to generate a new expected
output file. Don't forget to desk check the generated output to make sure
! it's really what you expected to find! To run a single test after modifying
! a module, simply run regrtest.py without the -g flag:
./python Lib/test/regrtest.py test_spam.py
--- 82,104 ----
If you are writing test cases for module spam, you need to create a file
! in .../Lib/test named test_spam.py. In addition, if the tests are expected
! to write to stdout during a successful run, you also need to create an
! expected output file in .../Lib/test/output named test_spam ("..."
! represents the top-level directory in the Python source tree, the directory
! containing the configure script). If needed, generate the initial version
! of the test output file by executing:
./python Lib/test/regrtest.py -g test_spam.py
! from the top-level directory.
Any time you modify test_spam.py you need to generate a new expected
output file. Don't forget to desk check the generated output to make sure
! it's really what you expected to find! All in all it's usually better
! not to have an expected-out file (note that doctest- and unittest-based
! tests do not).
!
! To run a single test after modifying a module, simply run regrtest.py
! without the -g flag:
./python Lib/test/regrtest.py test_spam.py
***************
*** 96,109 ****
./python Lib/test/test_spam.py
! To run the entire test suite, make the "test" target at the top level:
make test
! On non-Unix platforms where make may not be available, you can simply
! execute the two runs of regrtest (optimized and non-optimized) directly:
./python Lib/test/regrtest.py
./python -O Lib/test/regrtest.py
Test cases generate output based upon values computed by the test code.
--- 109,132 ----
./python Lib/test/test_spam.py
! To run the entire test suite:
+ [UNIX, + other platforms where "make" works] Make the "test" target at the
+ top level:
+
make test
+
+ {WINDOWS] Run rt.bat from your PCBuild directory. Read the comments at
+ the top of rt.bat for the use of special -d, -O and -q options processed
+ by rt.bat.
! [OTHER] You can simply execute the two runs of regrtest (optimized and
! non-optimized) directly:
./python Lib/test/regrtest.py
./python -O Lib/test/regrtest.py
+ But note that this way picks up whatever .pyc and .pyo files happen to be
+ around. The makefile and rt.bat ways run the tests twice, the first time
+ removing all .pyc and .pyo files from the subtree rooted at Lib/.
Test cases generate output based upon values computed by the test code.
***************
*** 173,177 ****
Each test case is different. There is no "standard" form for a Python
! regression test case, though there are some general rules:
* If your test case detects a failure, raise TestFailed (found in
--- 196,202 ----
Each test case is different. There is no "standard" form for a Python
! regression test case, though there are some general rules (note that
! these mostly apply only to the "classic" tests; unittest- and doctest-
! based tests should follow the conventions natural to those frameworks):
* If your test case detects a failure, raise TestFailed (found in
***************
*** 213,224 ****
file support), even if all the required modules are available.
- * findfile(file) - you can call this function to locate a file somewhere
- along sys.path or in the Lib/test tree - see test_linuxaudiodev.py for
- an example of its use.
-
* verbose - you can use this variable to control print output. Many
modules use it. Search for "verbose" in the test_*.py files to see
lots of examples.
* use_large_resources - true iff tests requiring large time or space
should be run.
--- 238,267 ----
file support), even if all the required modules are available.
* verbose - you can use this variable to control print output. Many
modules use it. Search for "verbose" in the test_*.py files to see
lots of examples.
+ * verify(condition, reason='test failed'). Use this instead of
+
+ assert condition[, reason]
+
+ verify() has two advantages over assert: it works even in -O mode,
+ and it raises TestFailed on failure instead of AssertionError.
+
+ * TESTFN - a string that should always be used as the filename when you
+ need to create a temp file. Also use try/finally to ensure that your
+ temp files are deleted before your test completes. Note that you
+ cannot unlink an open file on all operating systems, so also be sure
+ to close temp files before trying to unlink them.
+
+ * sortdict(dict) - acts like repr(dict.items()), but sorts the items
+ first. This is important when printing a dict value, because the
+ order of items produced by dict.items() is not defined by the
+ language.
+
+ * findfile(file) - you can call this function to locate a file somewhere
+ along sys.path or in the Lib/test tree - see test_linuxaudiodev.py for
+ an example of its use.
+
* use_large_resources - true iff tests requiring large time or space
should be run.
***************
*** 259,260 ****
--- 302,330 ----
where coverage is adequate or lacking and write test cases to beef up the
coverage.
+
+
+ Some Non-Obvious regrtest Features
+
+ * Automagic test detection: When you create a new test file
+ test_spam.py, you do not need to modify regrtest (or anything else)
+ to advertise its existence. regrtest searches for and runs all
+ modules in the test directory with names of the form test_xxx.py.
+
+ * Miranda output: If, when running test_spam.py, regrtest does not
+ find an expected-output file test/output/test_spam, regrtest
+ pretends that it did find one, containing the single line
+
+ test_spam
+
+ This allows new tests that don't expect to print anything to stdout
+ to not bother creating expected-output files.
+
+ * Two-stage testing: To run test_spam.py, regrtest imports test_spam
+ as a module. Most tests run to completion as a side-effect of
+ getting imported. After importing test_spam, regrtest also executes
+ test_spam.test_main(), if test_spam has a "test_main" attribute.
+ This is rarely needed, and you shouldn't create a module global
+ with name test_main unless you're specifically exploiting this
+ gimmick. In such cases, please put a comment saying so near your
+ def test_main, because this feature is so rarely used it's not
+ obvious when reading the test code.
Index: test_difflib.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_difflib.py,v
retrieving revision 1.2
retrieving revision 1.3
diff -C2 -r1.2 -r1.3
*** test_difflib.py 2001/05/23 01:45:19 1.2
--- test_difflib.py 2001/05/23 07:46:36 1.3
***************
*** 1,2 ****
import doctest, difflib
! doctest.testmod(difflib, verbose=0)
--- 1,3 ----
+ from test_support import verbose
import doctest, difflib
! doctest.testmod(difflib, verbose=verbose)
Index: test_doctest.py
===================================================================
RCS file: /cvsroot/python/python/dist/src/Lib/test/test_doctest.py,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -r1.1 -r1.2
*** test_doctest.py 2001/02/10 01:36:47 1.1
--- test_doctest.py 2001/05/23 07:46:36 1.2
***************
*** 1,2 ****
import doctest
! doctest.testmod(doctest, verbose=1)
--- 1,3 ----
+ from test_support import verbose
import doctest
! doctest.testmod(doctest, verbose=verbose)