unittest: Calling tests in liner number order

[Fuzzyman]

On May 25, 3:13 pm, Roy Smith <r... at panix.com> wrote:
> In article
> <f838bc64-a4ef-4770-b139-01284e0f8... at t54g2000hsg.googlegroups.com>,
>  John Roth <johnro... at gmail.com> wrote:
>
> > I really don't care what the OP does in his own projects. My objection
> > is that, if it goes into the standard library, is that it passes a
> > signal that it's good practice to allow dependencies between tests. It
> > most definitely is _not_ good practice.
>
> The OP stated that he wants the tests run in a given order.  People are
> assuming that's because he has dependencies between his tests.  Maybe he
> just wants them run in order because it's easier to understand the output
> in a certain order.

No, we're pointing out that running tests in a specific order can
introduce hidden dependencies without you being aware of it. Whilst
this is already the case with unittest, further enshrining it in the
standard library is a bad idea.

As mentioned elsewhere, providing a better reporting mechanism is
probably the way to get better understandable output.

Micahel Foord
http://www.ironpythoninaction.com/

>
> For example, I'm currently working on some low-level data marshalling code
> (in another language).  This is the sort of stuff which the Python struct
> module handles -- converting between internal form and network
> representation.  
>
> There is a logical hierarchy of complexity.  Handling characters is easier
> than handling ints, which is easier than floats, which is easier than
> doubles, and so on (arrays, sets and other composite types).  If I was
> porting this to a new platform, I would want the unit tests to run in a
> logical order of simple to complex.  If some assortment of tests failed,
> you want to start debugging the problem on the least complex data types.  
> If the tests run in a specific order, it's easier to see where to start.
>
> It's not that any test depends on any other test to run, in the sense that
> there's some shared state between them.  It's just that from a human
> factors point of view, there's a logical order to run them in.
>
> In fact, from a protocol point of view, some of the types really do depend
> on each other.  We send counted strings, for example, so we can't send a
> string until we know how to send an int (for the string length).  If the
> first test that fails is the string test, I know right off that the problem
> is not in how we send ints, because that test ran already and it passed.
>
> Earlier, I gave another example of wanting tests to be run in the same
> order as some externally controlled set of functional requirements.  Again,
> not because the tests have inter-dependencies, but because it just makes it
> easier to interpret the results.
>
> Don't assume inter-test dependencies (which I agree are a Bad Thing) is the
> only reason you want to run tests in a specific order.