[Python-Dev] Using more specific methods in Python unit tests

[Terry Reedy]

On 2/15/2014 1:12 PM, Serhiy Storchaka wrote:
> Many Python tests were written a very long time before the unittest,
> using simple asserts. Then, when they have been ported to the unittest,
> asserts were replaced with the assert_ method and then with assertTrue.
> The unittest has a number of other methods to check for and report
> failure, from assertEqual, to more specific assertIs, assertIn,
> assertIsInstance, etc, added in 2.7. New methods provide better
> reporting in case of failure.

Failure of assertTrue reports 'False is not true'.

> I wrote a large patch which modifies the tests to use more specific
> methods [1]. Because it is too large, it was divided into many smaller
> patches, and separate issues were opened for them. At the moment the
> major part of the original patch has already been committed. Many thanks
> to Ezio for making a review for the majority of the issues. Some changes
> have been made by other people in unrelated issues.
>
> Although Raymond approved a patch for test_bigmem [2], his expressed the
> insistent recommendation not to do this. So I stop committing new
> reviewed patches. Terry recommended to discuss this in Python-Dev. What
> are your thoughts?
>
> [1] http://bugs.python.org/issue16510
> [2] http://bugs.python.org/issue20547

After thinking about Raymond's objections and checking
http://docs.python.org/3/library/unittest.html#test-cases
and noting Ezio's explicit approval and others tacit approval (by lack 
of objection), I think you should continue and finish.

The reasons for not making making style changes in public stdlib modules 
are that we want people to import and use them, have promised stability 
absent notice otherwise, and because it is reasonably possible to make 
unintended semantic changes that might pass unnoticed by incomplete tests.

The case is different on all counts for test.text_xyz modules. They are 
private application modules, not to be imported (except to run them), 
and subject to change without notice. Nearly all the changes proposed 
are from assertTrue to various specialized assertXs with documented 
equivalences to assertTrue. I presume you made the 1-to-1 replacements 
with a script using capturing regexes, with a check for false 
replacements such as might occasionally happen due to an operator being 
inside a string. If we are ever to make the replacements, doing so 
mechanically and in bulk is easier and safer than doing them one at a 
time by hand.  This includes reviewing.

-- 
Terry Jan Reedy