[issue25423] Deprecate benchmarks that execute too quickly

[Antoine Pitrou]

Antoine Pitrou added the comment:

I haven't seen that:

$ ./perf.py -f -b silent_logging `which python2.7` `which python3.4`
INFO:root:Automatically selected timer: time
Running silent_logging...
INFO:root:Running `/home/antoine/.local/bin/python3.4 ./performance/bm_logging.py -n 5 --timer time no_output`
INFO:root:Running `/usr/bin/python2.7 ./performance/bm_logging.py -n 5 --timer time no_output`

Report on Linux fsol 3.19.0-30-generic #34-Ubuntu SMP Fri Oct 2 22:08:41 UTC 2015 x86_64 x86_64
Total CPU cores: 4

### silent_logging ###
Min: 0.056003 -> 0.075588: 1.35x slower
Avg: 0.056043 -> 0.076493: 1.36x slower
Significant (t=-38.61)
Stddev: 0.00003 -> 0.00118: 47.2238x larger


In any case, if it runs too fast, we should simply increase the number of iterations.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25423>
_______________________________________