[pytest-dev] Future of pytest-xdist/execnet?

[Anthony Sottile]

Just my 2c, I think I might would be valuable to create two new plugins:
- one which handles local concurrency via multiprocessing (pipes, shmem,
etc.)
- one which handles remote concurrency (sockets, client/server, etc.)

from a usage standpoint I expect the 99% to be the former suggested plug-in
(for current usage of xdist).  At least for me all of my (admittedly
little) usage is of vanilla xdist on the local machine. I also think by
separating concerns the two plugins could focus more on implementing their
core functionality (and therefore do a better job of it).

only if there's very clear identical behavior should we consider making a
shared library or including in core.

I definitely think there's a lot of opportunities here to improve on the
status quo (and provide some significant value for those currently using
xdist). I've collected a few (I don't want to call them complaints) words
from a few heavy users (pip, cryptography to name a few) where I think a
new approach could provide significant improvements

while they're both essentially distributed systems problems, I think the
local vs. remote plugins can probably take slightly different
approaches/trade-offs on reliability (especially with local multiprocessing
likely being much more reliable)

I'd be happy to help work/help on this, though I probably don't have the
cycles to properly "own" such an effort

Anthony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20191024/fd35105f/attachment.html>