[pytest-dev] Crowdfunding a pytest plugin

[Tibor Arpas]

Hi Florian

Longer ago you had some good points about selecting only tests affected by
recent changes as a basic functionality for our plugin - testmon.

We did more testing and thinking about these difficult questions and I
think they will not be that bad.


> I also thought about using something like coverage to only run affected
> tests. How much testing did you do in your prototype?


E.g. pytest test suite itself runs pretty nicely with testmon.


> Do you keep track of (pip/setuptools) installed packages (versions)?


Not yet, but I'm pretty sure a simple detection of changes under sys.path
that would trigger a full test run should be easy to implement and quick
enough.

Did you try changes at various levels like new files, modules, classes,
> metaclasses, mocks etc? Python is very dynamic as you most likely know :)
>

* files: sure
* modules, classes, metaclasses, mocks:
Unless I'm missing something this is not necessary. All of the dynamic
constructs are created deterministically in projects py files. If none of
the files involved in execution of a test is changed, then all of the
dynamic constructs will be the same in the next test run. Of course I might
be missing somethings, so if somebody has a good case please, let us know.

These things cause trouble for coveragepy (multiprocessing and gevent stuff
is beeing solved): http://nedbatchelder.com/code/coverage/trouble.html

These are my thoughts about the dependencies of a test:
https://github.com/tarpas/testmon#thoughts

Thanks for help,

Tibor
testmon - make your tests a breeze to execute http://igg.me/at/testmon


>
> Regards,
> Florian Schulze
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20150313/47b55c65/attachment.html>