[pytest-dev] Need access to parameter IDs during pytest_collection_modifyitems

[Pete Savage]

Hi all,

Forgive this seemingly useless requirement. We have the requirement to collect the parameters which
are used in a parametrized test to pass to a test case results system.

Currently I am pulling out the parameters using the item.callspec.params except there are issues
where the parameter is an object and has a differing repr than that id that was used in the idlist.
To keep things consistent, I would like to tell the test case result system the ID of the parameter
that was used for the test, if one is available and if not to fall back to the object.

This is basically being used to describe a whole set of tests during a --collect-only that _should_
run based on parametrization and passing those into the results system, so that they can be
populated across multiple smaller runs with the actual results.

The problem is this information is not available on the item object. Here is what IS available:

Consider the following contrived test

---
import pytest


@pytest.mark.parametrize('num', [1, 2, 3], ids=["one", "two", "three"])
@pytest.mark.parametrize('letter', ['a', 'b', 'c'], ids=["A", "B", "C"])
def test_something(num, letter):
    print num, letter
---

During the modifyitems hook I can access the following:

* item.callspec.params
  * gives me a dict {'num': 1, 'letter': 'a'}) - but this doesn't give me IDs
* item.callspec.indices
  * gives me a dict {'num': 0, 'letter': 0}) - this is great that if I had the ID list for
    each param I could look up the right one
* item.callspec._idlist
  * gives me a list ['A', 'one']) - this is great except I don't know which ID is for which
    parameter
* item._getobj().parametrize
  * gives me <MarkInfo 'parametrize' args=('letter', ['a', 'b', 'c'], 'num', [1, 2, 3])
    kwargs={'ids': ['one', 'two', 'three']}> - which unfortunately is missing the kwargs
    entry for the IDs for the second parameter)

Seems like a problem with the MarkInfo item?

Is there any way I can construct this information?

Thanks,

-- 
Pete Savage
Principal Quality Engineer
Red Hat UK
psavage at redhat.com     IM: psav
redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
Twitter: @redhatway | Instagram: @redhatinc | Snapchat: @redhatsnaps