[Python-Dev] Use regrtest.py --timeout on buildbots

[Victor Stinner]

Le jeudi 31 mars 2011 à 18:35 +0200, Victor Stinner a écrit :
> Hi,
> 
> I just added a --timeout option to Lib/test/regrtest.py: if a test (one
> function, not a whole file) takes more than TIMEOUT seconds, the
> traceback is dumped and it exits. I tested it on 3 buildbots with a
> timeout of 5 minutes and it worked as expected: see #11727 for
> examples. 
> 
> It would be nice to have this feature enabled on all buildbots.

I enabled this timeout with a timeout of 15 minutes. Thanks to this
timeout, I have know a traceback for the strange test_threadsignals
hang:
http://bugs.python.org/issue11738

But I got also 3 buildbots (2 FreeBSD, 1 Solaris) failing on: test_io,
test_subprocess, test_signal.

I changed the default timeout to 30 minutes. The timeout was too long to
catch a test_ssl failure on Windows 7 (which uses a timeout of 20
minutes), but also long enough to avoid false positive on Solaris. The 2
FreeBSD buildbots still fails (test_io, test_socket).

I am not sure yet that the failures with timeouts of 15 or 30 minutes
are just false positive. For example, test_interrupted_write_buffered()
in test_io was interrupted after 30 minutes on "x86 FreeBSD 3.x" whereas
this test takes less then 5 seconds on my Linux box (and on my FreeBSD
VM).

Anyway, I am happy to have a working tool to get more information on
buildbot hang. At least, it is possible to enable it temporary to try to
learn more on a failure.

The timeout protects also the buildbot against bugs (hang or infinite
loop) in Python or in the test suite: bugs are detected earlier.

Victor