[Cython] speed.pypy.org

[Robert Bradshaw]

On Fri, Apr 15, 2011 at 1:20 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Stefan Behnel, 11.04.2011 15:08:
>>
>> I'm currently discussing with Maciej Fijalkowski (PyPy) how to get Cython
>> running on speed.pypy.org (that's what I wrote "cythonrun" for). If it
>> works out well, we may have it up in a couple of days.
>
> ... or maybe not. It may take a little longer due to lack of time on his
> side.
>
>
>> I would expect that Cython won't be a big winner in this game, given that
>> it will only compile plain untyped Python code. It's also going to fail
>> entirely in some of the benchmarks. But I think it's worth having it up
>> there, simply as a way for us to see where we are performance-wise and to
>> get quick (nightly) feed-back about optimisations we try. The benchmark
>> suite is also a nice set of real-world Python code that will allow us to
>> find compliance issues.
>
> Ok, here's what I have so far. I fixed a couple of bugs in Cython and got at
> least some of the benchmarks running. Note that they are actually simple
> ones, only a single module. Basically all complex benchmarks fail due to
> known bugs, such as Cython def functions not accepting attribute assignments
> (e.g. on wrapping). There's also a problem with code that uses platform
> specific names conditionally, such as WindowsError when running on Windows.
> Cython complains about non-builtin names here. I'm considering to turn that
> into a visible warning instead of an error, so that the name would instead
> be looked up dynamically to let the code fail at runtime *iff* it reaches
> the name lookup.

Given the usefulness of the error, and the (relative) lack of issues
with it so far, I'd rather not turn it into only a warning by default
(though an option might be nice). Another option would be to whitelist
the presumably small, finite set of names that are platform-dependent.

> Anyway, here are the numbers. I got them with "auto_cpdef" enabled, although
> that doesn't even seem to make that a big difference. The baseline is a
> self-compiled Python 2.7.1+ (about a month old).

Cool.  So basically everything is faster, usually somewhere between a
50-100% improvement. There's lots of room for improvement, though a
JIT has a significant advantage that we don't get for untyped code.

- Robert


> ### ai ###
> Min: 0.402407 -> 0.362190: 1.1110x faster
> Avg: 0.408784 -> 0.366898: 1.1142x faster
> Significant (t=10.017195, a=0.95)
> Stddev: 0.00824 -> 0.00442: 1.8668x smaller
>
>
> ### chaos ###  (with a bug fixed in the benchmark itself)
> Min: 0.393362 -> 0.231932: 1.6960x faster
> Avg: 0.401941 -> 0.234089: 1.7170x faster
> Significant (t=36.128709, a=0.95)
> Stddev: 0.01004 -> 0.00267: 3.7538x smaller
>
>
> ### crypto_pyaes ###
> Min: 2.629560 -> 1.276433: 2.0601x faster
> Avg: 2.639409 -> 1.277742: 2.0657x faster
> Significant (t=368.652396, a=0.95)
> Stddev: 0.00812 -> 0.00153: 5.3215x smaller
>
>
> ### fannkuch ###
> Min: 1.512630 -> 0.853309: 1.7727x faster
> Avg: 1.522860 -> 0.860237: 1.7703x faster
> Significant (t=118.573908, a=0.95)
> Stddev: 0.00880 -> 0.00887: 1.0073x larger
>
>
> ### float ###
> Min: 0.452620 -> 0.343341: 1.3183x faster
> Avg: 0.475137 -> 0.349356: 1.3600x faster
> Significant (t=9.575876, a=0.95)
> Stddev: 0.02838 -> 0.00757: 3.7489x smaller
>
>
> ### go ###
> Min: 0.758998 -> 0.491929: 1.5429x faster
> Avg: 0.764110 -> 0.496518: 1.5389x faster
> Significant (t=90.848407, a=0.95)
> Stddev: 0.00400 -> 0.00523: 1.3096x larger
>
>
> ### nbody_modified ###
> Min: 0.399168 -> 0.197931: 2.0167x faster
> Avg: 0.401379 -> 0.203112: 1.9762x faster
> Significant (t=42.377829, a=0.95)
> Stddev: 0.00293 -> 0.01004: 3.4337x larger
>
>
> ### raytracesimple ###   (module renamed from "raytrace-simple")
> Min: 2.016425 -> 1.182970: 1.7045x faster
> Avg: 2.030030 -> 1.192164: 1.7028x faster
> Significant (t=78.219481, a=0.95)
> Stddev: 0.02184 -> 0.00983: 2.2211x smaller
>
>
> ### richards ###
> Min: 0.286723 -> 0.162430: 1.7652x faster
> Avg: 0.289933 -> 0.165193: 1.7551x faster
> Significant (t=52.898468, a=0.95)
> Stddev: 0.00392 -> 0.00352: 1.1127x smaller
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel
>