[pytest-dev] How can I programmatically add a fixture to a test

[Alessandro Amici]

All,

apparently I can answer my own question.

First of all, the `pytest_generate_tests` is the wrong place to replace the
test function as the metafunc object is not really used outside of the
method, only it's _calls attribute is.

Then, after digging quite a bit into the source code, I found a may to
manipulate the collected function before its arguments are extracted inside
`pytest_pycollect_makeitem`.

I'm working in this branch
https://github.com/nodev-io/pytest-nodev/tree/marker-interface but what
does the trick is the following:

def pytest_pycollect_makeitem(collector, name, obj):
    search_marker = getattr(obj, 'search', None)
    if search_marker and getattr(search_marker, 'args', []):
        target_name = search_marker.args[0]

        def wrapper(wish, monkeypatch, *args, **kwargs):
            if '.' in target_name:
                monkeypatch.setattr(target_name, wish, raising=False)
            else:
                monkeypatch.setattr(inspect.getmodule(obj), target_name,
wish, raising=False)
            return obj(*args, **kwargs)

        wrapper.__dict__ = obj.__dict__
        return list(collector._genfunctions(name, wrapper))

With this the following two tests are equivalent:

def test_parse_bool(wish):
    parse_bool = wish
    assert not parse_bool('false')
    [...]

@pytest.mark.search('parse_bool')
def test_parse_bool():
    assert not parse_bool('false')
    [...]

The way to programmatically add fixtures to a test function is to wrap it
inside `pytest_pycollect_makeitem`.

This is almost perfect, as with this method I can only wrap test function
with no funcargs.

Cheers,
Alessandro

On Mon, 14 Mar 2016 at 19:59 Alessandro Amici <alexamici at gmail.com> wrote:

> All,
>
> TL;DR - How do I programmatically (e.g. inside pytest_generate_tests) wrap
> a test function with a wrapper function that accept an additional fixture?
>
> Very long version
>
> I was on the verge to release version 1.0 of the pytest-nodev plugin [1]
> that enable test-driven search via a fixture or via a decorator, when I
> noticed that I could use the marker interface as a much nicer API, but I
> couldn't figure out the last bit I need.
>
> The context
>
> Following the example in the README assume I need to write a `parse_bool`
> function that robustly parses a boolean value from a string. Here is the
> test I intend to use to validate my own implementation once I write it:
>
>     def test_parse_bool():
>         assert not parse_bool('false')
>         assert not parse_bool('FALSE')
>         assert not parse_bool('0')
>         assert parse_bool('true')
>         assert parse_bool('TRUE')
>         assert parse_bool('1')
>
> I can "instrument" the test to use with pytest-nodev in two ways, first
> using the"wish" fixture explicitly:
>
>     def test_parse_bool(wish):     # <- here...
>         parse_bool = wish          # <- ... and here
>         assert not parse_bool('false')
>         assert not parse_bool('FALSE')
>         assert not parse_bool('0')
>         assert parse_bool('true')
>         assert parse_bool('TRUE')
>         assert parse_bool('1')
>
> the search is the executed with:
>
>     py.test test_parse_bool.py --wish-from-stdlib
>
> The test is run once for every function found in the standard library and
> the functions that make the test pass (the little gem that
> is distutils.util:strtobool) is presented in the result summary.
>
> This is working right now, but the user needs to modify its test in a non
> trivial way. For the curious I parametrize the metafunc inside
> pytest_generate_tests [2].
>
> So I'm experimenting with a following decorator interface:
>
>     import pytest_nodev
>
>     @pytest_nodev.search('parse_bool')
>     def test_parse_bool():
>         assert not parse_bool('false')
>         assert not parse_bool('FALSE')
>         assert not parse_bool('0')
>         assert parse_bool('true')
>         assert parse_bool('TRUE')
>         assert parse_bool('1')
>
> The "search" decorator takes the name of the tested object and uses the
> "monkeypatch" fixture together with the "wish" fixture to have the same
> result as above. This works right now and it is much easier for the users.
> The implementation is more complex though [3].
>
> This is still not perfect because you still need to prepare the test
> specifically to be used for a search with pytest_nodev.
>
> The best approach would be to use a marker:
>
>     import pytest
>
>     @pytest.mark.target('parse_bool')
>     def test_parse_bool():
>         assert not parse_bool('false')
>         assert not parse_bool('FALSE')
>         assert not parse_bool('0')
>         assert parse_bool('true')
>         assert parse_bool('TRUE')
>         assert parse_bool('1')
>
> The marker is a nice documentation and can be left even when pytest-nodev
> is not installed.
>
> The problem is I didn't find a way to use the marker to monkeypatch and
> parametrize the test. This is the closest I could get:
>
>     def pytest_generate_tests(metafunc):
>         search_marker = getattr(metafunc.function, 'search', None)
>         if not search_marker:
>             return
>
>         # setup the free variables for the wrapper closure
>         target_name = search_marker.args[0]
>         test_func = metafunc.function
>
>         def wrapper(wish, monkeypatch):
>             monkeypatch.setattr(target_name, wish)
>             return test_func()
>
>         # trying to make a hand-made test decorator
>         metafunc.function = wrapper
>
>         ids, params = make_wish_index(metafunc.config)
>         metafunc.parametrize('wish', params, ids=ids, scope='module')
>
> But this dies with:
>
> pytest_nodev/plugin.py:134: in pytest_generate_tests
>     metafunc.parametrize('wish', params, ids=ids, scope='module')
> ../mac-cpython3/lib/python3.5/site-packages/_pytest/python.py:1000: in
> parametrize
>     raise ValueError("%r uses no fixture %r" %(self.function, arg))
> E   ValueError: <function pytest_generate_tests.<locals>.wrapper at
> 0x1064b7400> uses no fixture 'wish'
>
> Obviously I didn't manage to simulate [2] inside the
> pytest_generate_tests. I guess that I need to tell the metafunc that the
> function now use the wish and the monkeypatch fixtures.
>
> So my question is: can I programmatically simulate adding a decorator with
> arbitrary fixtures to a test? Is there a better way to do it?
>
> I understand the use case is complex and in case anybody is willing to
> help I'll do my best to ease their work, like creating a dedicated branch
> in the github repo.
>
> Thanks,
> Alessandro
>
> [1]. https://github.com/nodev-io/pytest-nodev
> [2].
> https://github.com/nodev-io/pytest-nodev/blob/master/pytest_nodev/plugin.py#L121
>
> [3].
> https://github.com/nodev-io/pytest-nodev/blob/master/pytest_nodev/__init__.py#L12
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20160315/236bab61/attachment.html>