[Python-Dev] test_descrtut failing

[Tim Peters]

[Jack Jansen]
> ...
> The fun thing is that if I run it completely standalone (by dragging
> test_descrtut.py to the interpreter, the test can handle this
> situation) it works fine!  So, there must be something wrong with
> either the test framework (or maybe this re-import trick?) that
> doesn't work as expected on the Mac.

It's that the name of a module depends on how it's first imported, and
that's got nothing to do with Macs or the framework.  It does have to do
with the exact way in which you run the tests; apparently nobody else runs
tests that way, or nobody else runs tests at all <wink>.

BTW, the "re-import trick" is neither a trick nor a re-import:  doctest
takes a module object as input, and importing a module is the *obvious* way
to get a module object.  The module may or may not have been imported
before, but even it was it wasn't imported before *by* the doctested module;
so it's no more of a re-import than two modules both deciding to import,
say, math or sys.  Unfortunately, unlike as for math or sys, the name of a
module in a package is fuzzy.

> Maybe test_regrtest can be thought which tests use unittest or
> doctest, and not try to be helpful in those cases?

Sure -- but it's not in my plans.

> For now I've just put a note in the readme file for MacPython that
> this test is expected to fail, but I'd like to fix it eventually, of
> course.

Or you can include the hacked version I just checked in (forcing "test."
prefix gibberish to show up in the module name no matter how it's run, via
Guido's suggested trick).

> I have another gripe with the new unittest stuff (this is the first
> time I've seen the doctest thing, never knew it was there!), and
> that's that most of the test failures are difficult to
> interpret. Whereas the old-style tests simply "print math.sin(math.pi)"
> and tell you they expected the output to be 0 but got -1 in stead the
> unittest-based tests often don't give that information.
> Most of the unittest-based tests use only failUnless/assert_ in stead
> of the higher-level functions, i.e. test_time uses things like
>         self.assert_(time.ctime(self.t)
>                  == time.asctime(time.localtime(self.t)))
> where the test results would be a lot easier to interpret if they used
>       self.assertEqual(time.ctime(self.t),
> 		time.asctime(time.localtime(self.t)),
> 		"ctime(T) != asctime(localtime(T)")

The same gripe is just as true of many of the straight regrtests, using
test_support.verify() lazily, as in

verify(D.goo(1) == (D, 1))
verify(str(c1).find('C instance at ') >= 0)
verify(d == {})
verify(lines == ["A\n", "B\n", "C", "D\n", "E\n", "F"])
verify(len(m) == 2*PAGESIZE)
verify(str(e) == 'Truncated input file')
verify(re.split("(?::*)", ":a:b::c") == ['', 'a', 'b', 'c'])

and on & on & on.  There are over 900 (static) calls of verify(), and it's
rare to see the optional "explanation" argument.  Many of those in turn were
originally lazy uses of the assert stmt (also skipping the "reason" clause).

This is human nature <0.1 wink>, and it played a large role in doctest's
design.  For example, a doctest for the last one there could look like:

"""
>>> import re
>>> re.split("(?::*)", ":a:b::c")
['', 'a', 'b', 'c']
"""

and if it fails you get a message showing you the example, the expected
output, and the output you actually got.  Like

*****************************************************************
Failure in example: re.split("(?::*)", ":a:b::c")
from line #2 of dc
Expected: ['', 'a', 'b', ':c']
Got: ['', 'a', 'b', 'c']
*****************************************************************

Less typing and better error reports.  Multiline "expected" and "got" stuff
work just as slick, which regrtest doesn't handle well (it compares output
at a low per-.write() level, not at line granularity; that's why when a
straight regrtest fails, you'll sometimes see a string of "expected"
characters that come out of the middle of one of the expected-output lines).