[pytest-dev] Parametrized autouse fixtures
Thomas De Schampheleire
patrickdepinguin at gmail.com
Sat Mar 17 15:52:05 EDT 2018
Hi,
I now prepared the following PR for Kallithea to use these
parametrized fixtures:
https://bitbucket.org/conservancy/kallithea/pull-requests/389/tests-vcs-automatic-parametrization/diff
but I have a question:
The parametrized fixture is set up here in a base class:
https://bitbucket.org/conservancy/kallithea/pull-requests/389/tests-vcs-automatic-parametrization/diff#Lkallithea/tests/vcs/base.pyT55
and has two parameters 'hg' and 'git'.
Most test classes indeed need to be parametrized this way, but some
should only be run for one of the parameters, either hg or git.
So far I handled this with a fixture inside the special classes,
skipping tests as required:
https://bitbucket.org/conservancy/kallithea/pull-requests/389/tests-vcs-automatic-parametrization/diff#Lkallithea/tests/vcs/test_git.pyT649
+ @pytest.fixture(autouse=True)
+ def _skip_unsupported_scm(self):
+ if self.backend_class.scm != 'git':
+ pytest.skip('Unsupported scm for this test: %s' %
self.backend_class.scm)
but I wonder if there is a better way to do this, avoiding following two issues:
1. the above fixture needs to be duplicated for each test class. For a
fixture that does not need 'self', one could put it in conftest.py and
use @pytest.mark.usefixtures() to avoid the duplication, but it is my
understanding that this is not possible if you need access to self.
2. the 'skips' are shown in the test overview, which is not very
useful in this case IMO because it is normal that they are skipped,
i.e. it is not due to an environment mismatch, or a temporarily
skipped testcase. It will always be skipped.
Thanks,
Thomas
2018-03-13 23:05 GMT+01:00 Thomas De Schampheleire <patrickdepinguin at gmail.com>:
> Thanks a lot, this did help me very well!
>
> /Thomas
>
> 2018-03-12 22:16 GMT+01:00 Bruno Oliveira <nicoddemus at gmail.com>:
>> Hi Thomas,
>>
>> It seems the problem is that you are mixing xunit-style fixtures (by
>> implementing a setup_class classmethod) and pytest-style fixtures: they
>> currently don’t play well together, setup_* methods execute before all other
>> fixtures (see #517). There are plans to fix that, but this scheduled for 3.6
>> only.
>>
>> But there’s an easy workaround, just change your “setup_class” method into a
>> proper fixture instead:
>>
>> class TestArchivesTestCaseMixin:
>>
>> @classmethod
>> @pytest.fixture(autouse=True, params=['hg', 'git'], scope='class')
>> def _configure_backend(cls, request):
>> backend_alias = request.param
>> Backend = vcs.get_backend(backend_alias)
>> ...
>>
>> Hope that helps.
>>
>> Cheers,
>> Bruno.
>>
>>
>> On Mon, Mar 12, 2018 at 5:48 PM Thomas De Schampheleire
>> <patrickdepinguin at gmail.com> wrote:
>>>
>>> Hello,
>>>
>>> The Kallithea project is a repository hosting and review system,
>>> currently supporting git and hg. We currently have some test cases
>>> that need to be run for these two version control systems.
>>>
>>> Previously this was done with some Python magic, which was now
>>> simplified and made more explicit in commit:
>>>
>>> https://kallithea-scm.org/repos/kallithea/changeset/45a281a0f36ff59ffaa4aa0107fabfc1a6310251
>>>
>>> but I assume it can be made more automatic with pytest fixtures. I was
>>> thinking to use an autouse, parametrized fixture to set the backend to
>>> 'hg' and 'git' respectively. But I can't make it work.
>>>
>>> Here is the change I did on top of the mentioned commit:
>>>
>>> diff --git a/kallithea/tests/vcs/base.py b/kallithea/tests/vcs/base.py
>>> --- a/kallithea/tests/vcs/base.py
>>> +++ b/kallithea/tests/vcs/base.py
>>> @@ -5,6 +5,7 @@ InMemoryChangeset class is working prope
>>> import os
>>> import time
>>> import datetime
>>> +import pytest
>>>
>>> from kallithea.lib import vcs
>>> from kallithea.lib.vcs.nodes import FileNode
>>> @@ -27,6 +28,11 @@ class _BackendTestMixin(object):
>>> """
>>> recreate_repo_per_test = True
>>>
>>> + @pytest.fixture(autouse=True,
>>> + params=['hg', 'git'])
>>> + def set_backend_alias(cls, request):
>>> + cls.backend_alias = request.param
>>> +
>>> @classmethod
>>> def get_backend(cls):
>>> return vcs.get_backend(cls.backend_alias)
>>> diff --git a/kallithea/tests/vcs/test_archives.py
>>> b/kallithea/tests/vcs/test_archives.py
>>> --- a/kallithea/tests/vcs/test_archives.py
>>> +++ b/kallithea/tests/vcs/test_archives.py
>>> @@ -14,7 +14,7 @@ from kallithea.tests.vcs.base import _Ba
>>> from kallithea.tests.vcs.conf import TESTS_TMP_PATH
>>>
>>>
>>> -class ArchivesTestCaseMixin(_BackendTestMixin):
>>> +class TestArchivesTestCaseMixin(_BackendTestMixin):
>>>
>>> @classmethod
>>> def _get_commits(cls):
>>> @@ -95,11 +95,3 @@ class ArchivesTestCaseMixin(_BackendTest
>>> def test_archive_prefix_with_leading_slash(self):
>>> with pytest.raises(VCSError):
>>> self.tip.fill_archive(prefix='/any')
>>> -
>>> -
>>> -class TestGitArchive(ArchivesTestCaseMixin):
>>> - backend_alias = 'git'
>>> -
>>> -
>>> -class TestHgArchive(ArchivesTestCaseMixin):
>>> - backend_alias = 'hg'
>>>
>>>
>>>
>>> but when running this I get:
>>>
>>> $ pytest kallithea/tests/vcs/test_archives.py
>>> Test session starts (platform: linux2, Python 2.7.14, pytest 3.4.2,
>>> pytest-sugar 0.9.1)
>>> benchmark: 3.1.1 (defaults: timer=time.time disable_gc=False
>>> min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10
>>> warmup=False warmup_iterations=100000)
>>> rootdir: /home/tdescham/repo/contrib/kallithea/kallithea-review,
>>> inifile: pytest.ini
>>> plugins: sugar-0.9.1, localserver-0.4.1, benchmark-3.1.1
>>>
>>>
>>> ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
>>> ERROR at setup of TestArchivesTestCaseMixin.test_archive_zip[hg]
>>> ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
>>> kallithea/tests/vcs/base.py:70: in setup_class
>>> Backend = cls.get_backend()
>>> kallithea/tests/vcs/base.py:38: in get_backend
>>> return vcs.get_backend(cls.backend_alias)
>>> E AttributeError: type object 'TestArchivesTestCaseMixin' has no
>>> attribute 'backend_alias'
>>>
>>>
>>> 7% ▊
>>>
>>> ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
>>> ERROR at setup of TestArchivesTestCaseMixin.test_archive_zip[git]
>>> ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
>>> kallithea/tests/vcs/base.py:70: in setup_class
>>> Backend = cls.get_backend()
>>> kallithea/tests/vcs/base.py:38: in get_backend
>>> return vcs.get_backend(cls.backend_alias)
>>> E AttributeError: type object 'TestArchivesTestCaseMixin' has no
>>> attribute 'backend_alias'
>>>
>>>
>>> 14% █▌
>>>
>>> [..]
>>>
>>>
>>>
>>> So the parametrization seems to work because each test is run twice,
>>> but I can't seem to find the right way to set the backend_alias, such
>>> that a later call from a classmethod works correctly.
>>>
>>> I found some possibly useful magic by nicoddemus but could not make
>>> that work either:
>>> https://github.com/pytest-dev/pytest/issues/2618#issuecomment-318519875
>>>
>>> Is it possible to achieve what I want, without changing each test
>>> method to take a fixture explicitly?
>>>
>>> Thanks,
>>> Thomas
>>> _______________________________________________
>>> pytest-dev mailing list
>>> pytest-dev at python.org
>>> https://mail.python.org/mailman/listinfo/pytest-dev
More information about the pytest-dev
mailing list