[pytest-dev] Introduction

[Isaul Vargas]

Hi everyone! My name is Isaul Vargas and I work as a software tester.
I have been using py.test since 2013 and I really like the fixture
interface of
Py.test it's a powerful abstraction!

One of the libraries I use to test software is Selenium (and Appium for
mobile devices),
and one indispensable plug-in I use is pytest-xdist. I love how xdist lets
me run tests in
parallel, and initially it gave me the impression that tests were
distributed among processes
in a queue.

However to my surprise, test distribution is batched and uneven; this is
disappointing because
if you're running slow functional, end-to-end tests, it's going to take a
long time to run a big batch of tests,
if the tests are distributed in an uneven manner.

In my own test run of Appium tests, using -n 12, The initial batch of tests
runs 12 processes; then
when one or two tests finish, another one or two tests are dispatched.
However after this initial batch,
I only see about 6-8 active processes running. This is very disappointing,
and I thought Python had
an edge here, compared to other parallel test runners that only batch tests
according to the number of
processes/threads specified. (like Junit,TestNG on the Java side)

So, I've decided to volunteer my time to help improve the Pytest-xdist
plugin so that we have a
true queue based system of test distribution so that every slot is always
busy running tests, and save
developers a ton of time!

I would like to thank Ronny for taking the time to explain how the current
system works, and what
changes are needed to make this happen. I am motivated to get this issue
fixed; so that we are the
first test runner (that I know of) to use a queue based system that doesn't
simply batch tests.

For those wondering why batching is bad, when you have tests that take 2-3
minutes to complete,
plus a few tests that take 5 minutes to complete; if you batch tests such
that 1 workers gets all the 5 minutes tests
and another workers gets all the short tests then the total run time is
almost the same as running all the large tests
in serial.

I look forward to improving this aspect of pytest-xdist, and I am highly
motivated to get this done. I look forward
to diving deep into the code and learn the internals of pytest.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pytest-dev/attachments/20160125/6b2fb3f6/attachment.html>