replacing __dict__ with an OrderedDict

[Lie Ryan]

On 01/07/2012 04:20 AM, Ian Kelly wrote:
> On Fri, Jan 6, 2012 at 10:01 AM, Lie Ryan<lie.1296 at gmail.com>  wrote:
>> That unittest executes its tests in alphabetical order is implementation
>> detail for a very good reason, and good unittest practice dictates that
>> execution order should never be defined (some even argued that the execution
>> order should be randomized). If the test runner turns out to execute tests
>> concurrently, that should not cause problems for a well-designed test.
>> Displaying the test results in a more convenient order for viewing is what
>> you really wanted in 99.99% of the cases.
>
> Randomizing the order is not a bad idea, but you also need to be able
> to run the tests in a consistent order, from a specific random seed.
> In the real world, test conflicts and dependencies do happen, and if
> we observe a failure, make a change, rerun the tests and observe
> success, we need to be able to be sure that we actually fixed the bug,
> and that it didn't pass only because it was run in a different order.
>
> Concurrent testing is a bad idea for this reason -- it's not
> repeatable (testing concurrency, OTOH, is a perfectly fine thing to be
> thinking about).

Concurrent testing is perfectly fine strategy in the case where you have 
thousands of tests and running them synchronously will just take too 
long. Certainly it makes it harder to repeat the test if there is any 
sort of dependency in the tests, but when you have the large number of 
tests, the benefit may exceeds the drawbacks.