unittest.Testsuite and execution order

[Chris Angelico]

On Thu, Apr 19, 2018 at 2:51 AM, Francesco Russo <francescor82 at gmail.com> wrote:
> My use case: my SUT is split into modules. Besides writing unit tests for
> each module, I want to write an integration test, and I also need to
> perform some actions between two calls to the SUT. In my case, the order of
> the execution is important.

In that case, make these tests into a single test. If you have to do
the steps together to correctly test it, they're not separate tests,
they're separate parts of the same test.

> class MyTestCode(unittest.TestCase):
>    def test_func_1(self):
>       # do something to test func_1 on the SUT
>       sut.func_1()
>       self.assert(...)
>
>    def perform_intermediate_step(self):
>       # do something between func_1 and func_2
>
>    def test_func_2(self):
>       # do something to test func_2 on the SUT
>       sut.func_2()
>       self.assert(...)

This is a bad idea. Each function that starts test_ should be
completely independent. You should be able to run any one of them on
its own (say, if you're trying to figure out why your latest change
caused a test failure), and it should have the same result.

Make it so that test_func_1 and test_func_2 are completely
independent, and then, if you need a single test that uses both, have
a test that calls on each function.

> Such an example works, today, since TestSuite uses a list, and addTest()
> appends to the list.
> My question is: is this something that I can rely on for the future? I
> definitely don't want to rely on the current implementation, unless I see
> it in the documentation.

I would say no, you can't rely on it. If you can't find it in the
docs, don't assume it's true. Test order randomization can be
controlled with a simple command line flag.

ChrisA