[Speed] steps to get pypy benchmarks running

[Geoffrey Sneddon]

On 13/02/18 21:52, Mark Shannon wrote:
> Hi,
> 
> On 13/02/18 14:27, Matti Picus wrote:
>> I have begun to dive into the performance/perf code. My goal is to get
>> pypy benchmarks running on http://speed.python.org. Since PyPy has a
>> JIT, the benchmark runs must have a warmup stage.
> 
> Why?
> The other interpreters don't get an arbitrary chunk of time for free, so
> neither should PyPy. Warmup in an inherent cost of dynamic optimisers.
> The benefits should outweigh the costs, but the costs shouldn't be ignored.

Both from startup performance and hot performance are interesting,
depending on use-case (for a long-running server, hot performance is
interesting; for a short-run script, including warmup is significant).
I'd suggest providing both figures for PyPy.

/gsnedders