[issue22813] No facility for test randomisation

[Michael Foord]

Michael Foord added the comment:

The point is that it is easy to have unintentional dependencies between tests. Test a sets up some state that test b relies on. This means that test b passes, so long as test a has already run. This is bad, tests should be isolated - it also means you can break test b when you change test a. Randomising test run order means you discover these unintentional dependencies earlier.

With test randomisation you ideally need the seed to be displayed as part of the test run, and you need to be able to run with a particular seed. This enables you to reproduce failures, or odd results, from any particular test run.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22813>
_______________________________________