[py-dev] new resource API documentation comments

[Brianna Laugher]

Hi,

I just spent some time reading the dev docs so these comments are just
based on the docs and not actually using the new API. In general it
looks pretty sensible.

- being able to have funcargs like funcargs directly is really nice, a
lot more obvious than calling request.getfuncargvalue('foo')
- why addfinalizer and not teardown?
-although I don't really know what cached_setup did, the trinity of
defining the scope, setup and teardown methods made sense to me. Now
the scope is in a decorator, the setup is implicitly the entire thing
that is happening and the teardown seems somewhat awkwardly tacked on.
- none of the "addfinalizer" examples take an argument, how would you
convert an old-style teardown method to that? e.g. we have a lot of
funcargs which do things like
    return request.cached_setup(setup=setup,
                                teardown=lambda obj: obj.close(),
                                scope='function')
- Sometimes things are referred to as "funcargs", sometimes they are
referred to as "injected resources". Is there any difference here? The
funcarg is the actual function and the injected resource is the
instance in a specific test function? I suggest to use the term
"funcarg" as much as possible as it is specific and a necessary
concept for using pytest with any depth.

Some of the following comments are fairly picky so feel free to ignore them.

funcargs.txt
line 118 - I think in this first incarnation of the smtp funcarg
(factory? what to call it now?), it doesn't actually need to take a
testcontext, right?

line 527 "Parametrizing test functions" - may be worth having a simple
example showing a combination using both (test data) parametrization
and funcarg parametrization, to emphasise how they are differently
useful. Using a database as an example of funcarg parametrization is
good, maybe better than values like 1/2. I feel like parametrizing
tests (test data) is probably the more common use case and it is a
little buried amongst the heavy duty parametrized funcargs.

line 598 "Basic ``pytest_generate_tests`` example" - I think this is
not a very basic example! I think it is copied from parametrize.txt
page, where it might make more sense. Here is what I would consider a
basic example.

# code
def isSquare(n):
    n = n ** 0.5
    return int(n) == n

# test file
def pytest_generate_tests(metafunc):
    squares = [1, 4, 9, 16, 25, 36, 49]
    for n in range(1, 50):
        expected = n in squares
        if metafunc.function.__name__ == 'test_isSquare':
            metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected))


def test_isSquare(n, expected):
    assert isSquare(n) == expected

Well, this is so trivial you might bundle it all into a single test,
but it can be useful to have each case genuinely be a separate test.
You could also shoe-horn this data into a parametrize decorator I
suppose but it is nicer to have more space if your test data is more
complicated, to explain it.

I am starting to have a feeling that the way my project has been using
generate_tests is not the way everyone else uses it. In our
conftest.py one of the enterprising developers on the project (who got
us all onto py.test initially) put this:

def pytest_generate_tests(__multicall__, metafunc):
    """Supports parametrised tests using generate_ fns.
    Use multicall to call any other pytest_generate_tests hooks first.
    If the test_ fn has a generate_ fn then call it with the metafunc
to let it parametrise the test.
    """
    __multicall__.execute()
    name = metafunc.function.__name__.replace('test_', 'generate_')
    fn = getattr(metafunc.module, name, None)
    if fn:
        fn(metafunc)

this means I can simplify my example above to:

def generate_isSquare(metafunc):
    squares = [1, 4, 9, 16, 25, 36, 49]
    for n in range(1, 50):
        expected = n in squares
        metafunc.addcall(id=n, funcargs=dict(n=n, expected=expected))

maybe this is a useful pattern for others? or maybe we're doing it
wrong, I dunno :)

in funcarg_compare.txt
line 53 - (in old-style) "4. there is no way how you can make use of
funcarg factories in xUnit setup methods."

Is there an example of this? I would like to see it! This sounds like
a way for people with existing xUnit style test suites to more easily
convert to funcarg style.

Maybe with all the parametrized test examples it is a good idea to
show the verbose output, just to be more explicit about what is being
generated.

As an aside, there is a comment in example/markers.txt line 144 that
"You can use the ``-k`` command line option to only run tests with
names matching
the given argument". Whether it is an oversight in the functionality
or the docs, I have always found that -k also matches against markers
(py.test.mark decorators). (Currently using pytest 2.2.3)

in setup.txt
line 65 'old = tmpdir.chdir()'
- it's not really relevant to the example but I don't know what this
does, and I couldn't find out, so it's just a bit distracting.

thanks as always for providing this great library :)

cheers
Brianna

-- 
They've just been waiting in a mountain for the right moment:
http://modernthings.org/