[pytest-dev] Parametrized autouse fixtures

[Floris Bruynooghe]

Thomas De Schampheleire <patrickdepinguin at gmail.com> writes:

> Hi,
>
> I now prepared the following PR for Kallithea to use these
> parametrized fixtures:
> https://bitbucket.org/conservancy/kallithea/pull-requests/389/tests-vcs-automatic-parametrization/diff
> but I have a question:
>
> The parametrized fixture is set up here in a base class:
> https://bitbucket.org/conservancy/kallithea/pull-requests/389/tests-vcs-automatic-parametrization/diff#Lkallithea/tests/vcs/base.pyT55
> and has two parameters 'hg' and 'git'.
> Most test classes indeed need to be parametrized this way, but some
> should only be run for one of the parameters, either hg or git.
>
> So far I handled this with a fixture inside the special classes,
> skipping tests as required:
> https://bitbucket.org/conservancy/kallithea/pull-requests/389/tests-vcs-automatic-parametrization/diff#Lkallithea/tests/vcs/test_git.pyT649
> +    @pytest.fixture(autouse=True)
> +    def _skip_unsupported_scm(self):
> +        if self.backend_class.scm != 'git':
> +            pytest.skip('Unsupported scm for this test: %s' %
> self.backend_class.scm)
>
> but I wonder if there is a better way to do this, avoiding following two issues:
> 1. the above fixture needs to be duplicated for each test class. For a
> fixture that does not need 'self', one could put it in conftest.py and
> use @pytest.mark.usefixtures() to avoid the duplication, but it is my
> understanding that this is not possible if you need access to self.
> 2. the 'skips' are shown in the test overview, which is not very
> useful in this case IMO because it is normal that they are skipped,
> i.e. it is not due to an environment mismatch, or a temporarily
> skipped testcase. It will always be skipped.

Firstly a disclaimer that I haven't wrapped my head around all your code
or followed the previous history of this thread.  So my suggestion may
be somewhat off.

I'd be quite tempted to try and solve this with a mark.  E.g. you can
mark tests with @pytest.mark.vcs, or @pytest.mark.vcs('git', 'hg') or
@pytest.mark.vcs('git') or @pytest.mark.vcs('hg') - I imagine the first
two mean the same thing.  You can mark whole classes, modules or
individual tests.  You can probably even mix them and Ronny can
doubtless explain all the corner cases and pitfalls you encounter by
mixing these.  But that should probably not deter you too much.

Anyway, once your tests are marked you can either use this mark info
directly into your existing autouse fixture.  But as you say you'll then
get skips which really shouldn't have been tests at all.  So you can
also implement your own pytest_generate_tests hook which uses the mark
info to generate the correct tests only.  Then the autouse fixture only
needs to check if a tests needs a backend and if so initialise it.

I'll again add a disclaimer that this is what I'd try and build.  I
haven't done that, so this might break down somewhere.


Cheers,
Floris