[Python-Dev] Buildbots and regrtest timeout

[Paul Moore]

On 6 June 2011 11:29, Victor Stinner <victor.stinner at haypocalc.com> wrote:
> Stephan Krah asked me to change how the default timeout is defined for regrtest
> (issue #12250):
>
> "The implicit timeout in regrtest.py makes it harder to write automated
> test scripts for 3rd party modules. First, you have to remember to
> set --timeout=0 for long running tests. Then, you have to remember
> not to use the --timeout option when compiling --without-threads.
>
> I'd much prefer that there's no timeout unless explicitly specified.
> For the buildbots, I think this could be done in the Makefile."
>
> First I replaced the hardcoded constant in regrtest.py by a command line
> argument (--timeout=3600) in the TESTOPTS variable of the Makefile, so if you
> call regrtest directly (without make), there is no more default timeout.
>
> But today I saw a a buildbot timeout without any traceback: a possible hang in
> test_io on "x86 FreeBSD 7.2 3.x" buildbot, "command timed out: 3900 seconds
> without output". I realized that some buildbots (all buildbots?) override the
> TESTOPTS variable ("make buildbottest TESTOPTS= TESTPYTHONOPTS=" for "x86
> FreeBSD 7.2 3.x").
>
> I moved the timeout option from TESTOPTS to a new variable: TESTTIMEOUT, with
> a default value of 1 hour (3600 seconds). The timeout is now only used for
> "make buildbottest". Use TESTTIMEOUT=0 to disable completly regrtest timeout.
> The timeout argument if ignored (with a warning) if
> faulthandler.dump_tracebacks_later() function is missing (e.g. if Python is
> compiled without threads).
>
> Please, don't disable the TESTTIMEOUT option for your buildbot. You may want
> to use a shorter or longer timeout, it just have to be smaller than the
> buildbot timeout (3900 seconds by default, 1 hour 6 minutes, just a little bit
> more than regrtest timeout: 1 hour). If you want a timeout longer than 1 hour,
> you can to change TESTTIMEOUT and the buildbot timeout (I don't know how to
> configure the buildbot timeout).
>
> I didn't touch test, testall, testuniversal, quicktest make rules, which don't
> use the regrtest timeout anymore. I would prefer to use the same timeout
> options for all test rules in TESTOPTS, but it doesn't work because some
> buildbots do override TESTOPTS variable.

How does this impact Windows buildbots? As they don't use the
makefile, did you add an override to the Windows scripts as well, or
will WIndows now use the default of no timeout?

Paul.