[issue18232] running a suite with no tests is not an error

[Robert Collins]

New submission from Robert Collins:

In bug https://bugs.launchpad.net/subunit/+bug/586176 I recorded a user request - that if no tests are found, tools consuming subunit streams should be able to consider that an error.

There is an analogous situation though, which is that if discover returns without error, running the resulting suite is worthless, as it has no tests. This is a bit of a sliding slope - what if discover finds one test when there should be 1000's ? 

Anyhow, filing this because there's been a few times when things have gone completely wrong that it would have helped CI systems detect that. (For instance, the tests package missing entirely, but tests were being scanned in the whole source tree, so no discover level error occurred).

I'm thinking I'll add a '--min-tests=X' parameter to unittest.main, with the semantic that if there are less than X tests executed, the test run will be considered a failure, and folk can set this to 1 for the special case, or any arbitrary figure that they want for larger suites.

----------
messages: 191282
nosy: michael.foord, rbcollins
priority: normal
severity: normal
status: open
title: running a suite with no tests is not an error

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18232>
_______________________________________