[issue20910] Make sleep configurable in tests

[STINNER Victor]

STINNER Victor added the comment:

"How much this patch speeds up testing? Especially interesting results for medium-speed buildbots (about a hour)."

Quick benchmark, I only ran the test once. "Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz" (4 cores with HT: 8 logical cores) and 12 GB of RAM. I tested test_sleep.patch + more_reliable_tests.patch with TEST_SLEEP=1e-5 and TEST_SHORT_SLEEP=1e-6, 

Sequence tests
==============

Command: "time ./python -m test".

Original: 18 min 40 sec.

real	18m39.936s
user	10m43.139s
sys	1m1.410s

Patched patch: 7 min 52 sec (58% faster).

real	7m51.608s
user	5m1.324s
sys	0m13.376s

The speedup would be much lower in practice, a sleep of 10 us cannot be used for buildbots. But you may use such crazy sleeps can be used on your PC to test your patches faster.


Parallel tests
==============

Command: "time ./python -m test -j10"

Original: 3 min 21 sec

real	3m20.716s
user	18m17.689s
sys	0m45.073s

Patched patch: 3 min 12 sec *but 6 tests failed* (test_asyncore, test_ftplib, test_multiprocessing_fork, test_multiprocessing_forkserver, test_multiprocessing_spawn, test_threading).

real	3m11.723s
user	19m32.329s
sys	0m48.024s

Since the test suite failed on the patched Python, it's not possible to compare performances, but the total duration is very close to the original python. The speed up looks to be null.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue20910>
_______________________________________