[Python-Dev] Questions on unittest behaviour

[Jim Jewett]

Collin Winter:

> [ improvements to unittest]

These would all have been (mostly) reasonable when the module was
first added to the standard library.  But it has been there as an ugly
fragile unchanging beast for several years, and is ironically not well
endowed with unit tests.  I therefore think it might be too late for
these changes.

> 2) The TestLoader.testMethodPrefix attribute currently allows anything
> to be assigned to it, including invalid objects and the empty string.
>  While the former will cause errors ... you get things like assertEqual(),
> failUnlessEqual(), etc, when TestLoader.loadTestsFromTestCase() is run.

It makes the interface ugly, and learning harder, but for code this
old, either people catch it with the first actual run of their tests
suites, or a subclass is handling it (perhaps to trigger a default, or
to call functions through a foreign interface).

> 3) TestLoader.loadTestsFromTestCase() accepts objects that are not
> test cases and will happily look for appropriately-named methods on
> any object you give it. This flexibility should be documented, or
> proper input validation should be done (a bug fix for 2.5).

People are probably using this, so I think a deprecation in 2.6 is the
fastest it should change.

> 4) TestLoader.loadTestsFromName() (and by extension,
> loadTestsFromNames(), too) raises an AttributeError if the name is the
> empty string because -- as it correctly asserts -- the object does not
> contain an attribute named ''. I recommend that this be tested for and
> ValueError be raised (bug fix for 2.5).

This seems reasonable for new code, but not this late in 2.5 -- if
people are going through an auto-generated list of attributes, they
may well be catching AttributeError and not ValueError.

> 5) When TestLoader.loadTestsFrom{Name,Names}() are given a name that
> resolves to a classmethod on a TestCase subclass, the method is not
> invoked.  ...
> It is not documented which of these tests takes priority: is the
> classmethod "a test method within a test case class" or is it a
> callable? The same issue applies to staticmethods as well.

Documentation probably should get added.

-jJ