unittest: Calling tests in liner number order

[Fuzzyman]

On May 24, 2:44 pm, Roy Smith <r... at panix.com> wrote:
> In article
> <af66b0d4-f77d-4d54-8ee0-94592497d... at l64g2000hse.googlegroups.com>,
>
>  Fuzzyman <fuzzy... at gmail.com> wrote:
> > Also, like others, I have had wonderful experiences of trying to track
> > down test failures that depend on the order that tests run in. Having
> > interdependencies between tests is a recipe for madness...
>
> I agree that tests should not depend on each other, but sometimes it's
> still useful to have the tests run in a certain order for reporting
> purposes.
>
> If you're doing requirements tracking, it's nice to have the tests execute
> in the same order as the requirements are listed.  It makes interpreting
> the output easier.  Sure, you could give the test cases names like
> "test_fr17.3a" and write your own getTestCaseNames(), but just putting them
> into the file in the order you want them to run is easier.  And making
> things easy is what this is all about.

Whilst I understand your point, I think the danger is that you end up
with hidden dependencies on the test order - which you're not aware of
and that the tests never expose. Certainly layout your tests in a
logical order within the file, but I think you risk potential problems
by controlling the order they are run in. Other frameworks
specifically provide test order randomizers for this very reason.

A worthwhile question for the OP - your patch seems fairly simple. Is
it easy for you to extend unittest for your own testing needs by
subclassing?

Unittest should definitely be easy for people who *want* this to add
it to their own testing environment.


All the best,


Michael Foord
http://www.ironpythoninaction.com/