[Python-Dev] [ python-Bugs-1124637 ] test_subprocess is far too slow (fwd)

[Marcus Alanen]

Guido van Rossum wrote:

> The Python test suite already has a way (the -u flag) to distinguish
> between "regular" broad-coverage testing and deep coverage for
> specific (or all) areas. Let's keep the really long-running tests out
> of the regular test suite.
> 
> There used to be a farm of machines that did nothing but run the test
> suite ("snake-farm"). This seems to have stopped (it was run by
> volunteers at a Swedish university). Maybe we should revive such an
> effort, and make sure it runs with -u all.

Hello Guido and everybody else,

I hacked together a simple distributed unittest runner for our projects. 
Requirements are a NFS-mounted home directory across the slave nodes and 
SSH-based "automatic" authentication, i.e. no passwords or passphrases 
necessary. It officially works-for-me for around three hosts (see below) 
so that cuts the time down basically to a third (real-life example ~600 
seconds to ~200 seconds, so it does work :-). It also supports 
"serialized tests", i.e. tests that must be run one after the other and 
cannot be run in parallel.

http://mde.abo.fi/tools/disttest/

Comes with some problems; my blurb from advogato.org:
"""
Disttest is a distributed unittesting runner. You simply set the 
DISTTEST_HOSTS variable to a space-separated list of hostnames to 
connect to using SSH, and then run "disttest". The nodes must all have 
the same filesystem (usually an NFS-mounted /home) and have the Disttest 
program installed. You even gain a bit with just one computer by setting 
the variable to "localhost localhost". :-)

There are currently two annoying problem with it, though. For some 
reason, 1) the unittest program connecting to the X server sometimes 
fails to provide the correct authentication, and 2) sometimes the actual 
connection to the X server can't be established. I think these are 
related to 1) congestion on the shared .Xauthority file, and 2) a too 
small listen() queue on the forwarding port by the SSH daemon. Both 
problems show up when using too many (over 4?) hosts, which is the whole 
point of the program! Sigh.
"""

Error checking probably bad. Anyway, feel free to check it out, modify, 
comment or anything. We're thinking of checking the assumptions in the 
blurb above, but no timetable is set.

My guess is that the NFS-mounted home directory is the showstopper and 
people usually don't have lot's of machines hanging around, but that's 
for you to decide.

Disclaimer: I don't know anything of CPython development nor of the 
tests in the CPython test suite. ;-)

Best regards, and a big thank you for Python,
Marcus