[pytest-dev] Fixture ordering and scopes

[Bruno Oliveira]

Hi Floris,


On Fri, Mar 16, 2018 at 5:42 AM Floris Bruynooghe <flub at devork.be> wrote:

> Bruno Oliveira <nicoddemus at gmail.com> writes:
>
> > Hi everyone and Holger,
> >
> > Looking at the code below:
> >
> >
> > data = {}
> > @pytest.fixture(scope='session')def clean_data():
> >     data.clear()
> > @pytest.fixture(autouse=True)def add_data():
> >     data['value'] = 1
> > @pytest.mark.usefixtures('clean_data')def test_foo():
> >     assert data.get('value')
> >
> > Should test_foo fail or pass? Keep your answer in mind before
> > proceeding. :)
>
> Nice example.  To be honest, my first reaction was that this is probably
> undefined.
>
> Part of my unhelpful side also thinks this is horrible anyway so why
> should we help people write bad code?
>
> > I ask this because *unrelated* fixtures, i.e. which don’t depend on each
> > other, are executed in the order they are declared in the test function
> > signature, regardless of scope.
>
> That the order is defined *is* surprising to me.  And it honestly smells
> like an implementation detail.
>
> > I opened up a PR <https://github.com/pytest-dev/pytest/pull/3306> which
> > does sort parameters by scope while keeping the relative order of
> fixtures
> > of same scope intact, and the test suite passes without failures so if
> the
> > current behavior is by design there are not tests enforcing it. Using
> code
> > in the PR, then things might also be surprising:
> >
> > @pytest.fixture(scope='session')def s(): pass
> > @pytest.fixture(scope='function')def f(): pass
> > def test_foo(f, s):
> >     pass
> >
> > s and f are unrelated and execute in (f, s) order in master, but in (s,
> f)
> > order in my branch.
>
> This seems very reasonable to me, but then I never considered the order
> of arguments important at all.  I don't really mind introducing the
> patch you suggest.  But the pedant in me would still not want to make
> any more guarantees other then the scoping, i.e. if you have
> test_foo(f1, f2, s) then the order of f1,f2 should remain undefined
> despite that s is going to be called first.
>

While I agree with the sentiment to not try to make too much guarantees so
we can change things in the future if so desired, in this case I think
preserving the order of the arguments for fixtures of same scope makes
sense and is useful.

For example suppose you have two session scoped fixtures from different
libraries which don't know about each other:

* `log_setup` from `core_logging` setups the builtin logging module in a
certain way (logs to a central server for instance).
* `db_setup` from `db` configures the database for testing, and makes use
of the builtin logging to log where the database is being created, etc.

In my application tests, which use `core_logging` and `db`, I want to setup
the fixtures in the correct order and the canonical way for that would be
to define an autouse session-scoped fixture in my conftest.py:

@pytest.fixture(scope='session', autouse=True)
def setup_logging_and_db(log_setup, db_setup): pass

So the order is important here, and leaving it undefined will require
people to write this instead:

@pytest.fixture(scope='session', autouse=True)
def my_setup_logging(log_setup): pass

@pytest.fixture(scope='session', autouse=True)
def my_db_setup(my_setup_logging  ): pass

While it works, it feels like unnecessary boilerplate.

The example above is not something I made up btw, I encountered that
situation more than once at work: fixtures from unrelated projects that
should be used together in a certain order, and not by a design mistake,
but just how things are supposed to work.

> Would like to hear what people think about this matter, as I think it is
> an
> > important one, specially *Holger* if this is a design decision or just an
> > accident of implementation, and if we should change it.
>
> My biggest worry is that this is a slippery slope.  E.g. why isn't a
> session-scoped fixture initialised right at the start of the session?
> Once your mental model has adjusted to this the current behaviour is
> pretty natural (and probably why I don't see it as an issue).  But I
> accept I know probably too much about pytest and do think the PR might
> make things more intuitive for people.
>

I think the issue here is people getting tripped by it, and not the case
that they are having inter-dependencies between fixtures that are implicit,
but that they assume if a test function requests fixtures of different
scopes, the higher scopes will be instantiated first.

Cheers,
Bruno.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20180316/45466f30/attachment-0001.html>