Skipping test using unittest SkipTest and exit status

[Chris Angelico]

On Sun, May 15, 2016 at 12:28 AM, Ganesh Pal <ganesh1pal at gmail.com> wrote:
> The script show the below output , this looks fine for me. Do you see any
> problems with this ?
>
> gpal-ae9703e-1# python unitest1.py
> ERROR:root:Failed scanning
> E
> ======================================================================
> ERROR: setUpClass (__main__.ScanTest)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "unitest1.py", line 20, in setUpClass
>     raise unittest.TestCase.failureException("class setup failed")
> AssertionError: class setup failed
>
> ----------------------------------------------------------------------
> Ran 0 tests in 0.000s
>
> FAILED (errors=1)
>
> 2. I find  assert and raise RunTimeError also fitting my program  ,please
> suggest whats best  form unittest fixture point of view.
>
>
>          if not self.scan:
>             logging.error("Failed scanning ")
>             assert False, "Class setup failed skipping test"
>
>         if not self.scan:
>             logging.error("Failed scanning ")
>             raise  RuntimeError.
>
>  My overall ,idea is Setup class fails then don't run any of the next
> statements and exit the tests.

There are three quite different things happening in your three examples.

1) When a test *fails*, it means that the test ran successfully, but
didn't give the expected result. (This is sometimes a good thing - for
example, in test-driven development, you first create a test, then run
the test suite and see that it correctly fails, and then implement the
feature or fix, at which point the test suite will pass.) A test
failure happens when you call assertEqual with two unequal values, for
instance.

2) A test *exception* is generally a failure of the test suite itself.
It means that your code ran, but something went wrong, and an
exception was raised. That's what happens when you raise arbitrary
exceptions, including RuntimeError and AssertionError.

3) An *assertion failure* is (conceptually) an error in some
function's preconditions/postconditions, or a complete and utter
muck-up in implementation. It's something that should never actually
happen. It's something where you're legitimately allowed to skip over
every assert statement - in fact, that's exactly what happens when you
run Python in optimized mode.

Every exception has its purpose. With a lot of them, you can find out
that purpose by looking at its docstring:

>>> import builtins
>>> for name in dir(builtins):
...     obj = getattr(builtins, name)
...     if isinstance(obj, type) and issubclass(obj, BaseException):
...         print("%s: %s" % (obj.__name__, obj.__doc__.split("\n")[0]))
...
ArithmeticError: Base class for arithmetic errors.
AssertionError: Assertion failed.
AttributeError: Attribute not found.
BaseException: Common base class for all exceptions
BlockingIOError: I/O operation would block.
BrokenPipeError: Broken pipe.
BufferError: Buffer error.
BytesWarning: Base class for warnings about bytes and buffer related
problems, mostly
ChildProcessError: Child process error.
ConnectionAbortedError: Connection aborted.
ConnectionError: Connection error.
ConnectionRefusedError: Connection refused.
ConnectionResetError: Connection reset.
DeprecationWarning: Base class for warnings about deprecated features.
EOFError: Read beyond end of file.
OSError: Base class for I/O related errors.
Exception: Common base class for all non-exit exceptions.
FileExistsError: File already exists.
FileNotFoundError: File not found.
FloatingPointError: Floating point operation failed.
FutureWarning: Base class for warnings about constructs that will
change semantically
GeneratorExit: Request that a generator exit.
OSError: Base class for I/O related errors.
ImportError: Import can't find module, or can't find name in module.
ImportWarning: Base class for warnings about probable mistakes in module imports
IndentationError: Improper indentation.
IndexError: Sequence index out of range.
InterruptedError: Interrupted by signal.
IsADirectoryError: Operation doesn't work on directories.
KeyError: Mapping key not found.
KeyboardInterrupt: Program interrupted by user.
LookupError: Base class for lookup errors.
MemoryError: Out of memory.
NameError: Name not found globally.
NotADirectoryError: Operation only works on directories.
NotImplementedError: Method or function hasn't been implemented yet.
OSError: Base class for I/O related errors.
OverflowError: Result too large to be represented.
PendingDeprecationWarning: Base class for warnings about features
which will be deprecated
PermissionError: Not enough permissions.
ProcessLookupError: Process not found.
RecursionError: Recursion limit exceeded.
ReferenceError: Weak ref proxy used after referent went away.
ResourceWarning: Base class for warnings about resource usage.
RuntimeError: Unspecified run-time error.
RuntimeWarning: Base class for warnings about dubious runtime behavior.
StopAsyncIteration: Signal the end from iterator.__anext__().
StopIteration: Signal the end from iterator.__next__().
SyntaxError: Invalid syntax.
SyntaxWarning: Base class for warnings about dubious syntax.
SystemError: Internal error in the Python interpreter.
SystemExit: Request to exit from the interpreter.
TabError: Improper mixture of spaces and tabs.
TimeoutError: Timeout expired.
TypeError: Inappropriate argument type.
UnboundLocalError: Local name referenced but not bound to a value.
UnicodeDecodeError: Unicode decoding error.
UnicodeEncodeError: Unicode encoding error.
UnicodeError: Unicode related error.
UnicodeTranslateError: Unicode translation error.
UnicodeWarning: Base class for warnings about Unicode related problems, mostly
UserWarning: Base class for warnings generated by user code.
ValueError: Inappropriate argument value (of correct type).
Warning: Base class for warning categories.
ZeroDivisionError: Second argument to a division or modulo operation was zero.


Granted, not all of those docstrings tell us much (BufferError -
"Buffer error."), but it's a start. When you're picking something to
raise, try to be as appropriate as possible (don't signal a database
connection problem with TabError, for instance), and if nothing seems
particularly right, subclass the nearest match and make your own.
Neither RuntimeError nor AssertionError is the right way to say "my
test shouldn't be run"; you should decide whether you want the test to
fail or report an exception, and if you want the latter, check the
list above for something more appropriate.

Hope that helps!

ChrisA