[pytest-dev] Test discovery

[Florian Bruhin]

Hey Rob,

On Sat, Aug 01, 2020 at 11:16:52PM -0700, Rob McDonald wrote:
> According to Google, 'How does pytest discover tests' is an oft-asked
> question.  However most of those questions actually want to know 'What
> criteria does pytest use to discover tests'...
> 
> I'm working on a program that needs to scan a directory structure for
> Python code and then execute some of that code -- much like pytest does.
> So, I'd like to know _how_ does pytest discover (and run) tests.
> 
> I've searched around in the source for where this happens, but I haven't
> had any luck.  Much appreciated if someone can point me to the relevant
> bits of code?  Otherwise, any pointers on best practices and
> functions/methods to use for this sort of thing are appreciated.

Most of it happens in pytest's internal "python" plugin:

https://github.com/pytest-dev/pytest/blob/master/src/_pytest/python.py

For a simpler example how pytest's collection tree works, see:

https://docs.pytest.org/en/latest/reference.html#collection-hooks
https://docs.pytest.org/en/latest/example/nonpython.html

Finally, you might be interested in Oliver Bestwalter's talk "Abridged
metaprogramming classics - this episode: pytest":

https://github.com/obestwalter/abridged-meta-programming-classics

There, Oliver implements a small subset of pytest's "magic" from scratch.

Note, however, that there are various utility functions available in the stdlib
(inspect, importlib, pkgutils) which make those kinds of tasks easier, at least
with Python 3.5+. However, both pytest and Oliver's talk avoid them (because of
Python 2 support and because it was a fun limitation, respectively).

For a more straightforward implementation, you can roughly:

- Use pkgutil.walk_packages to find modules
- Use importlib.import_module to import them
- Use inspect.getmembers to get e.g. functions

I do the first two in my own project here:
https://github.com/qutebrowser/qutebrowser/blob/master/qutebrowser/extensions/loader.py

Florian

-- 
me at the-compiler.org (Mail/XMPP) | https://www.qutebrowser.org 
       https://bruhin.software/ | https://github.com/sponsors/The-Compiler/
       GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc
             I love long mails! | https://email.is-not-s.ms/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20200803/e17ab516/attachment.sig>