[pypy-dev] PyPy in the benchmarks game - yes or no?

[William ML Leslie]

On 5 April 2011 10:23, Isaac Gouy <igouy2 at yahoo.com> wrote:
> Hi
>
> A simple yes / no question.
>
> Do you want PyPy to be shown in the benchmarks game or not?
>
> Please consider the question amongst yourselves and then let me know.

There seems to have been general confusion here about what the
implementations of these benchmarks are supposed to represent.  Are
they to be representative of idiomatic code, or optimised for a
particular implementation?  Or something else entirely?  Since pypy
have generally tried to optimise for and encourage idiomatic python
usage, those benchmark implementations that go to great length to use
confusing and non-standard performance hacks represent neither the
performance of real-world code, nor what one can do with a specific
implementation.

If the answer is that they are going to be tuned to a particular
implementation and that implementation is not going to be ours, we
probably *could* live with that: realistically, the sort of code
people are applying pypy to occasionally contains performance hacks
that are no longer relevant and possibly detrimental.  But it does
seem to change the meaning of the benchmark, and it would be useful to
get some authoritative clarification on this before we consider it.

-- 
William Leslie