[issue10548] Error in setUp not reported as expectedFailure (unittest)

[Nick Coghlan]

Nick Coghlan added the comment:

While I agree with Robert's comments in general, the main question to be resolved here is the scope of the "expectedFailure" decorator.

Yes, it's applied specifically to the test execution phase when writing the code, but the question is how the following scenarios should be handled when the test is marked that way:

- setUp throws an exception
- test passes, tearDown or cleanUp throw an exception
- test fails, tearDown or cleanUp throw an exception
- test throws an exception, tearDown or cleanUp throw an exception

>From my perspective, those cases represent:

- error
- unexpected success OR error
- expected failure
- expected failure

If the test itself fails as expected, then I'm OK with cleanup code also failing as a consequence.

However, I'd also be OK with the simpler model that treats the decorator as covering the whole setUp/test/cleanUp/tearDown cycle, in which case an exception from *any* of those methods would be categorised as satisfying the "expected failure" decorator.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10548>
_______________________________________