[issue10548] Error in setUp not reported as expectedFailure (unittest)

[Michael Foord]

Michael Foord <michael at voidspace.org.uk> added the comment:

(made slightly redundant by Holger's comment but I'll continue anyway)

I think the issue is that setUp / tearDown are used for two different purposes.

The first is setting up and tearing down test infrastructure - where you do want to see to errors.

The other is for asserting pre and post conditions. If these are expected to fail (for whatever reason) then it may be perfectly reasonable to mark them as expectedFailure.

The fact that it was reported as a bug, and also that Antoine has requested being able to skip in a tear down (separate issue) shows that people are doing this.

So on the one hand - a small proportion of tests are marked expectedFailure and a very small subset of those might have a test infrastructure setup error. On the other hand for people who want setUp to test pre-conditions and want expectedFailure to work here will be completely unable to do this. It seems like not having consistent behaviour for expectedFailure will be more of a problem for those who want it than having it would be for those who don't need it.

As expectedFailure is not intended to be widely used anyway I would rather have consistency. It also allows the implementation to be simplified by unifying skip / expected fail / exception handling for all of setUp / tearDown / testMethod / cleanUp.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10548>
_______________________________________