[pytest-dev] solving the "too static" fixture scope problem

[holger krekel]

Hi Ronny,

On Fri, Oct 11, 2013 at 17:58 +0200, Ronny Pfannschmidt wrote:
> Hi Holger,
> 
> i think that just 'each' is too generic,
> i'd rather see something more specific like scope='session,function'

TLDR, i think you are tackling a different issue and one which
i am not sure we need to solve.

> i think its very important to be in control of the actual scopes used in a fixture

You already can, you just cannot specify a multitude of possible fixtures
within one fixture function.

> an example i have in mind is a semi-generic tmpdir fixture
> 
> @pytest.fixture(scope='session,function')
> def tmpdir(request, _pytest_basetmp):
>  if request.scope == 'session':
>    return _basetmp.ensure('session', dir=1)
>  elif request.scope == 'function':
>    return _basetmp.ensure('tests', dir=1)\
>      .make_numbered_dir(request.function__name__)

In my mail the use case was that each fixture/test receiving
"mytmpdir" (working title for the new builtin fixture) gets a fresh
new directory which it does _not_ share with any other fixture function.  
Such "helper" fixture functions are scope-agnostic, they don't care
about which scope they are requested in, and will perform their
finalization (if any) within the same scope as requested.  

By contrast, your example kind of extends the current API and thus
one would expect that the "tmpdir" is shared on each scope.

> or a little more detailed idea about databases:
> 
> @pytest.fixture(scope='session,function'):
> def db_connection(request):
>  if db_transactions_nested and request.scope=='session':
>    conn = connect(...)
>    schema.create_all(bind=conn)
>    dbsetup.install_initial_data(bind=conn)
>    return conn
>  elif db_transactions_nested and request.scope=='function':
>    conn = request.get_fixture('db_connection', scope='session')
>    transaction = conn.begin_nested()
>    request.addfinalizer(transaction.rollback)
>    return conn
>  elif request.scope=='function':
>    conn = connect(...)
>    schema.create_all(bind=conn)
>    dbsetup.install_initial_data(bind=conn)
>    return conn
>  else:
>    pyyest.fail('unexpected scope state')
> the key point for me is being explicit about scopes when declaring fixtures and requesting them.

This i would term a "nested resources" use case.  You can already solve
this today by writing something (slightly simplified) like this:

@pytest.fixture(scope="session")
def db_connection_session(request):
    conn = connect(...)
    schema.create_all(bind=conn)
    dbsetup.install_initial_data(bind=conn)
    return conn

@pytest.fixture(scope="function")
def db_connection(db_connection_session):
    transaction = conn.begin_nested()
    request.addfinalizer(transaction.rollback)
    return conn

Which is very explicit.  Users "db_connection_session" and
"db_connection" know exactly what they are getting and the
current fixture-scope-mismatch-detection machinery makes
sure you don't use the wrong one in many cases.

My original consideration really focuses around providing
fixtures which are never cached and thus can remain ignorant
on caching scope.  Examples this is useful for:

- mytmpdir (a fresh empty directory whenever requested)
- monkeypatch (a fresh new instance/finalization at the same scope
  as the requestor)

And a new one would be (i have this in several of my projects
but with hacks):

- Popen(...): subprocess Popen but making sure that all started
  subprocesses are terminated within the scope requested.
  E.g. if i use Popen in a module-scoped fixture, at the end of
  the module the subprocess will be killed.

cheers,
holger