[Speed] Experiences with Microbenchmarking

[Maciej Fijalkowski]

On Fri, Feb 12, 2016 at 7:00 PM, Armin Rigo <arigo at tunes.org> wrote:
> Hi Paul,
>
> On Fri, Feb 12, 2016 at 5:00 PM, Paul <paul at paulgraydon.co.uk> wrote:
>>> PyPy typically needs more than 2000 iterations to be warmed up.
>>
>> Same goes for the JVM. Off the top of my head it doesn't even start marking a method as hot until around 10,000 iterations (at which point it'll start to do the first stage of optimisations). If you're below that threshold you're dealing with pure interpreter performance.
>
> Ew, it's even longer than PyPy :-)
>
> In the PyPy case, the number 2000 is particularly bad, because the JIT
> starts after 1039 iterations.  It also adds a few extra paths
> afterwards, starting maybe around ~400-500 extra iterations (as a mean
> value).  Each time, the JIT produces more machine code
> and there is a relatively important pause.  So 2000 is close to the
> worst case: even running 2000 purely-interpreted iterations would be
> faster.
>
>
> A bientôt,
>
> Armin.
> _______________________________________________
> Speed mailing list
> Speed at python.org
> https://mail.python.org/mailman/listinfo/speed

Armin, those are "2000 iterations of a benchmark" and not "2000
iterations of a loop". A lot of those are pypy benchmarks, just run
longer