unittest.Testsuite and execution order

[Chris Angelico]

On Fri, Apr 20, 2018 at 3:01 PM, Francesco Russo <francescor82 at gmail.com> wrote:
> On 18/04/18 20:26, Chris Angelico wrote:
>> This is a bad idea. Each function that starts test_ should be
>> completely independent. You should be able to run any one of them on
>> its own (say, if you're trying to figure out why your latest change
>> caused a test failure), and it should have the same result.
>>
>> Make it so that test_func_1 and test_func_2 are completely
>> independent, and then, if you need a single test that uses both, have
>> a test that calls on each function.
>
> I'm not sure I understand you here.
> I understood that (besides, or instead of, making an integration test by
> making those tests into one test, as you wrote above) I could make a
> test for func_2 making it independent from func_1, for example this way:
>
> class MyTestFunc2(unittest.TestCase):
>    def setUp(self):
>       # Prepare preconditions for func_2
>
>    def test_func_2(self):
>       sut.func_2()
>       self.assert(...)
>
> Such a test case wouldn't even need a test suite.
> Is this what you meant?

What I mean is that test_func_1 and test_func_2 should be able to pass
or fail regardless of whether the other has been run or not. That kind
of independence. If you then want to write an integration test that
verifies that data created by func_1 can be read by func_2, that is a
separate test.

> The official "unittest" web pages for Python 2 and 3 say this, for the
> TestSuite class:
> *This class represents an aggregation of individual tests cases and test
> suites*
> saying nothing about the order. But the docstring of the TestSuite class
> says:
> *It will run the individual test cases in the order in which they were
> added, aggregating the results*
> Can I consider the docstring an official documentation as well?
>

That's something for other people to answer; I don't know whether
TestSuite is a replaceable class. I'm not sure what the mechanics are
for test randomization, but it is most definitely a thing.

ChrisA