[issue38477] magiccube2x2 permutations 28% slower with Python 3.8.0rc1 vs 3.7

[Gregory P. Smith]

Gregory P. Smith <greg at krypto.org> added the comment:

If I compile my own 3.7.5 and 3.8.0 both with --enable-optimizations, I am unable to reproduce this.

greg at zoonaut:~/sandbox/python/cpython/lh$ ../b37/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 75.92
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b38/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 84.53
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b37/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 79.90
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b38/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 84.19
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b37/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 86.49
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b37/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 83.95
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b38/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 78.85
greg at zoonaut:~/sandbox/python/cpython/lh$ ../b38/python permutations2x2 --max_depth 6 | egrep "^# Total_Time"
# Total_Time: 78.49

which if you add up the 3.7 and 3.8 times over their four runs each, they both come out to ~81 seconds average.  So I don't see a regression.

I'd say that benchmark is unstable.  Also, making sure you're comparing identically prepared builds of cpython rather than something made by different distro package owners.

----------
nosy: +gregory.p.smith -inada.naoki
resolution: not a bug -> 
stage: resolved -> 
status: closed -> open

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue38477>
_______________________________________