[pytest-dev] parametrization ids simplification/generalization

[holger krekel]

On Tue, Oct 22, 2013 at 19:17 +1100, Brianna Laugher wrote:
> So if you had this:
> 
> @py.test.mark.parametrize(('a', 'b', 'expected'), [
>     (1, 1, 2),
>     (1, 0, 1),
>     (1, -1, 0),
>     ])
> def test_addition(a, b, expected):
>     assert a + b == expected
> 
> what would the generated IDs be?
> 
> test_addition[a0-b0-expected0]
> test_addition[a1-b1-expected1]
> test_addition[a2-b2-expected2]

Yes.

> That just seems really pointless to me. If you're not going to display the
> values at all, why bother including each parameter's name?
> 
> I was thinking
> 
> test_addition[0]
> test_addition[1]
> test_addition[2]

That's an option i guess.  However, i'd rather like to see 
argument names, because there are other ways to paraemtrize and
they combine, i.e. at the fixture function.  If you have:

    @pytest.fixture(params=[1,2,3])
    def arg(request):
        return request.param

    @pytest.mark.parametrize("otherarg", ["a", "b"])
    def test_func(otherarg, arg):
        ...

then test ids like:

    test_func[0]
    test_func[1]
    test_func[2]
    test_func[3]
    test_func[4]
    test_func[5]

would not really give much of a clue whereas these ids:

    test_func[otherarg0-arg0]
    test_func[otherarg1-arg0]
    ...

would give more of a clue, wouldn't they?

> (again, as it already is if you parametrize using metafunc.addcall),
> although I accept that is rather brief, maybe better as something like
> 
> test_addition[param0]
> test_addition[param1]
> test_addition[param2]
> 
> but basically I don't see the benefit of replicating how many parametrized
> values there are, in each test id.
> 
> "-k arg0" is still going to be a pretty useless specification, just think
> if you have multiple parametrized tests in a single file. I think whatever
> solution here is not going to help the fact that you cannot specify an
> entire test name as in "-k 'test_addition[param0]'" - which I think will be
> fixed separately, as we discussed on issue #357.

Are all of your tests going to use "arg" as the argument name? 
I usually use less generic names and thus e. g. "-k http_address0" is pretty
specific.

That aside, are you aware you can always specify a testid path without
using -k? For example:

    py.test test_module::test_func[otherarg0-arg0]

You get such IDs by specifying "-rf" which lists IDs of failing tests
at the end and we could add "-r." to mean to also show IDs for passing tests
as well.

best,
holger