[pytest-dev] Remove invocation fixtures from 3.0?

[Bruno Oliveira]

I guess it’s better to take them out and reintroduce them in 3.1 with
the rough edges smoothened out than breaking stuff now.

Thanks everyone for the feedback.

While further debugging the issue, I managed to isolate the problem in a
small reproducible case, similar to what is found in qutebrowser’s test
suite. Problem is, I can’t explain the behavior I’m seeing, both in master
and in the features branches.

Consider these files (mind the directory layout):

# ---------- contents of tests/conftest.py ----------import pytest
@pytest.yield_fixture(scope='session')def app(app):
    yield 'customapp'
@pytest.yield_fixture(scope='session', autouse=True)def httpbin(app):
    yield
# ---------- contents of tests/test_dummy.py ----------import pytest
@pytest.mark.parametrize('i', range(10))def test_dummy(i, request):
    finalizers = request._fixturemanager._arg2fixturedefs['app'][0]._finalizer
    assert len(finalizers) == 1
# ---------- contents of ./conftest.py ----------import pytest
@pytest.yield_fixture(scope='session')def app():
    yield 'app'

Running this produces (cutting out some of the output):

$ py.test
============================= test session starts =============================
platform win32 -- Python 3.5.0, pytest-2.9.3.dev0, py-1.4.31, pluggy-0.3.1
rootdir: X:\pytest\tmp\bug, inifile: pytest.ini
plugins: hypothesis-3.1.0, xdist-1.14
collected 10 items

tests\test_dummy.py FFFFFFFFFF

================================== FAILURES ===================================
________________________________ test_dummy[0] ________________________________

i = 0, request = <FixtureRequest for <Function 'test_dummy[0]'>>

    @pytest.mark.parametrize('i', range(10))
    def test_dummy(i, request):
        finalizers =
request._fixturemanager._arg2fixturedefs['app'][0]._finalizer
>       assert len(finalizers) == 1
E       assert 2 == 1
E        +  where 2 = len([<function
call_fixture_func.<locals>.teardown at 0x034C3198>, <bound method
FixtureDef.finish of <FixtureDef name='app' scope='session'
baseid='tests' >>])

tests\test_dummy.py:7: AssertionError
________________________________ test_dummy[1] ________________________________

i = 1, request = <FixtureRequest for <Function 'test_dummy[1]'>>

    @pytest.mark.parametrize('i', range(10))
    def test_dummy(i, request):
        finalizers =
request._fixturemanager._arg2fixturedefs['app'][0]._finalizer
>       assert len(finalizers) == 1
E       assert 3 == 1
E        +  where 3 = len([<function
call_fixture_func.<locals>.teardown at 0x034C3198>, <bound method
FixtureDef.finish of <FixtureDef name='ap...ession' baseid='tests' >>,
<bound method FixtureDef.finish of <FixtureDef name='app'
scope='session' baseid='tests' >>])

tests\test_dummy.py:7: AssertionError

(and it continues...)
________________________________ test_dummy[9] ________________________________

i = 9, request = <FixtureRequest for <Function 'test_dummy[9]'>>

    @pytest.mark.parametrize('i', range(10))
    def test_dummy(i, request):
        finalizers =
request._fixturemanager._arg2fixturedefs['app'][0]._finalizer
>       assert len(finalizers) == 1
E       assert 11 == 1
E        +  where 11 = len([<function
call_fixture_func.<locals>.teardown at 0x034C3198>, <bound method
FixtureDef.finish of <FixtureDef name='ap...n' baseid='tests' >>,
<bound method FixtureDef.finish of <FixtureDef name='app'
scope='session' baseid='tests' >>, ...])

tests\test_dummy.py:7: AssertionError
========================== 10 failed in 0.06 seconds ==========================

As can be seen here, the number of finalizers registered for the FixtureDef
"app" just grows as the number of tests grows. The same can be said of the
“next” app fixture in the chain (._arg2fixturedefs['app'][1]).

Can anybody with some knowledge on the fixture mechanism explain the above
behavior and if it was intentional?

Cheers,
Bruno.
​

On Mon, Aug 15, 2016 at 4:22 PM Florian Bruhin <me at the-compiler.org> wrote:

> Hey,
>
> * Bruno Oliveira <nicoddemus at gmail.com> [2016-08-15 01:15:14 +0000]:
> > I've worked on the weekend on the issue discovered by Florian on the
> > qutebrowser test suite, and unfortunately I think the current
> > implementation will have to be re-thought because it contains a serious
> > problem regarding finalizers when fixtures are being overwritten.
>
> Thanks for all the debugging and all the other great work you do to
> get 3.0 out! :)
>
> > Because redesigning it might take a awhile, I suggest we remove
> invocation
> > fixtures for now so we can make the 3.0 release. We can then work this
> week
> > on the remaining issues[1] and make a release candidate by the middle of
> > the week, aiming for the official release by the end of the week.
> >
> > How does that sound?
>
> I agree with the other Flori.* people :D
>
> I guess it's better to take them out and reintroduce them in 3.1 with
> the rough edges smoothened out than breaking stuff now.
>
> Florian
>
> --
> http://www.the-compiler.org | me at the-compiler.org (Mail/XMPP)
>    GPG: 916E B0C8 FD55 A072 | http://the-compiler.org/pubkey.asc
>          I love long mails! | http://email.is-not-s.ms/
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20160816/17386e04/attachment-0001.html>