[pytest-dev] Configurable collection roots and reusable testsuites - Brainstorming

[holger krekel]

Hi Ronny, Ionel, all,

On Mon, Jan 25, 2016 at 00:46 +0100, Ronny Pfannschmidt wrote:
> Hi everyone,
> 
> a topic thats gripping since quite a while, is wanting to have reusable
> testsuites,
> so plugins/extensions that implement a behaviout,
> can import and use integration tests from a installed package,
> and use them in a different configuration

That's already possible to a degree.

If you hg-checkout https://bitbucket.org/hpk42/devpi and then look at

    devpi/server/test_devpi_server/functional.py

which is included from

    devpi/server/test_devpi_server/test_views.py

and also from the client side (from the installed devpi-server package):

    devpi/client/testing/test_functional.py

you'll note that the same tests run against different versions of a
fixture, i.e. "mapp" which gives high level access to devpi behaviour.
ASFAIK this pattern is also used by other projects.

I was sometimes thinking that some way to tell pytest more dynamically
what to collect, i.e. adding something to the test collection would
be nice.  You can already do it by hooking into the plugin collection 
hooks but that's a bit low-level as you need to tie into the collection
tree carefully.  The hard part of "it would be nice" here is to write
down a suggestion in forms of documentation on what API/semantics exactly 
we want to add and then go for implementing it.

best,
holger


> another detail that plays into that is certain plugins needing different
> collections as well
> an example is the doctest-module, which should scan for doctests in the
> installed modules
> that where imported, and not the ones in the working directory.
> 
> unfortunately its hard to find a good way to express those configured
> roots for collection,
> since the ini format on its own is rather limited, and i would like to
> avoid introducing sub-languagess
> 
> -- Ronny
> _______________________________________________
> pytest-dev mailing list
> pytest-dev at python.org
> https://mail.python.org/mailman/listinfo/pytest-dev
> 

-- 
about me:    http://holgerkrekel.net/about-me/
contracting: http://merlinux.eu