[pytest-dev] how to fail on exceptions in other threads and destructors

[holger krekel]

On Mon, Jun 03, 2013 at 09:32 -0700, Nikolaus Rath wrote:
> On 06/01/2013 06:42 PM, Adam Goucher wrote:
> >> I would like to make sure that a test fails when it results in an
> >> uncaught exception - even if the exception happens in a destructor or in
> >> a separate thread.
> >>
> >> What's the best way to do this?
> >>
> >> My idea is to modify sys.excepthook such that it keeps a list of all the
> >> exceptions encountered during a test run. Then, at the end of every
> >> test, I could check this list and raise an exception if there were any
> >> calls to sys.excepthook.
> >>
> >> However, I'm not quite sure how to do this properly without having to
> >> request an explicit fixture in every test. Is there a way to have
> >> initialization and finalization code run for every test function? And
> >> what kind of exception would I need to raise in the finalization code
> >> for py.test to recognize that it is the test that failed (rather than a
> >> bug in the finalization code)?
> >
> > IIRC, I think what you want to do is look at the pytest_runtest_makereport hook. It gets called at each part of the execution flow. You would then look at call.when to determine when the thing that went wrong, went wrong.
> > 
> > https://github.com/Element-34/Py.Saunter-Examples/blob/master/picpaste/webdriver/conftest.py#L34 is kinda an example. Though in my case I ignore the setup and teardown bits.
> 
> 
> That looks close to what I want to do, thanks for the link!
> 
> However, I have some trouble understanding what's actually happening here:
> 
> Am I correct that, despite its name, the pytest_runtest_makereport hook
> is called not just after the test has been teared down, but also after
> setup and calling? Or is it called before setup, calling and teardown?

yes, pytest_runtest_makereport is called for each of the three 
setup/call/teardown phases of running a test.  The phase is indicated
in ``call.when``.

> Also, what exactly is the __multicall__ argument? According to
> http://pytest.org/latest/plugins.html#py-test-hook-reference,
> pytest_runtest_makereport accepts just two (item, call) parameters...?

"__multicall__" is missing docs because it wasn't yet part of the
official API.  But i consider it now part of the official API FWIW :)

Any hook implementation can accept ``__multicall__`` in addition 
to the hook-specified parameters.  ``__multicall__.execute()`` allows
to call the pending hook implementations, thus allowing to wrap values.
Concretely, the following pattern allows to decorate/modify values 
from other hooks:

    @pytest.mark.tryfirst   # tell pytest to run this hook impl early
    def pytest_SOMEHOOK(..., __multicall__):
        result = __multicall__.execute()  # execute all other hook impls
        # decorate/process result
        return result

cheers,
holger




> 
> Best,
> Nikolaus
> _______________________________________________
> Pytest-dev mailing list
> Pytest-dev at python.org
> http://mail.python.org/mailman/listinfo/pytest-dev
>