[pypy-dev] use as benchmark pypy vs python if you please

[Armin Rigo]

Hi Dima,

On Mon, Aug 4, 2014 at 5:04 PM, Dima Tisnek <dimaqq at gmail.com> wrote:
> Attached is n-queens solver (pardon my naive algorithm), it runs:
> python 2.7.6: 17s
> pypy 2.4.0 alpha: 23s
> same nojit: 32s
>
> I've tried similar-looking algorithm for another problem before, and has
> similar results -- somehow pypy was slower.
>
> feel free to investigate / tweak or even use on speed.pypy.org

So, it took us more than one year, but now I finally figured it out.
The reason it is slower in PyPy is because sets recompute the items'
hash much more often than in CPython.  I fixed it in the branch
'keys_with_hash' (which is just too late for pypy 2.6.1).  Some
microbenchmarks are 2x or 3x faster now.

In your code, it shows up as the _diff() function, which returns "a -
b" where a and b are sets of complicated objects.  Actually, there are
always disjoint sets, so the original code might be a bit buggy :-)
But the _diff() operation is now twice faster.  I get the following
times on your nq.py example:

    python 2.7.3: 15.5s
    pypy 2.6.1 in keys_with_hash: 10.9s
    same with '--jit off': 20.3s


A bientôt,

Armin.