[py-dev] proposal: chaining pytest_generate_tests calls

[Ronny Pfannschmidt]

Hi,

after some refactoring in the tests for the new serializer and some chat
with Benjamin it got appearant that some form of chaining the
pytest_generate_tests is helpfull and necessart.

Since there are various details lurking around,
this is a initial try to collect the needs for propperly chaining
generate_tests calls.
I will handle each aspect in a separate section.
For each aspect i will add all items i currenly consider
intresting/usefull, that might be incomplete and/or more than
reasonable.

Necessary Scopes
----------------

conftest/plugin
    Required to allow comfortable whole-project generators
package/module
    the same goes for sub-groups of the tests
functions
    doing special cases at the module level hook tends to be a mess
    see the current serializer tests for reasons


Additional dimensions of variation that could proof usefull
------------------------------------------------------------

funcargs
  choose test generators based on the arguments a function explicitly
takes, 
  implicitly used funcargs via request.getfuncargvalue cant be
considered there
* marks
  using decorators to add values and the way to add them to the call


Chaining order
--------------

conftest plugin -> package -> subpackage -> module -> function


building the test id's
-------------------------

for just chaining the tests one can simply append to a tuple of id's for
each successive add_call invokation per hook

if variation on funcargs is considered one could show those like
additional keyword parameters

funcarg generators shouldn't support chaining themself, just use the
closest matching test generator

dealing with the param argument of addcall
-------------------------------------------

* provide the value of the most close scope to request objecs 
* previous values of params could be accessible via linking to the
precedessor of the current call or by providing a list ordered by scope
* for funcarg variation it might be helpfull to supple the funcarg
function with the param the funcarg related generator created


dealing with the funcargs argument of addcall
---------------------------------------------

* values from more close scope should override vales for less close
scope
* warnings could be printed for name-clashes/overrides


how to actually combine the generators
--------------------------------------

Note: early never released versions of the generate_tests feature
provided implied combinations of all matching test generators, however
that was fragile and got dumped for 'match most closest hook'

i thenk we should prefer a explicit combination where for each addcall
of an outer scope, the complete generate_tests function of the inner
scope has to be run

this results in a clean tree of invocations and better controll over the
actual combinations


regards Ronny Pfannschmidt