[Speed] Experiences with Microbenchmarking

[Armin Rigo]

Hi Paul,

On Fri, Feb 12, 2016 at 5:00 PM, Paul <paul at paulgraydon.co.uk> wrote:
>> PyPy typically needs more than 2000 iterations to be warmed up.
>
> Same goes for the JVM. Off the top of my head it doesn't even start marking a method as hot until around 10,000 iterations (at which point it'll start to do the first stage of optimisations). If you're below that threshold you're dealing with pure interpreter performance.

Ew, it's even longer than PyPy :-)

In the PyPy case, the number 2000 is particularly bad, because the JIT
starts after 1039 iterations.  It also adds a few extra paths
afterwards, starting maybe around ~400-500 extra iterations (as a mean
value).  Each time, the JIT produces more machine code
and there is a relatively important pause.  So 2000 is close to the
worst case: even running 2000 purely-interpreted iterations would be
faster.


A bientôt,

Armin.