[Python-Dev] Proposal for a common benchmark suite

[M.-A. Lemburg]

Stefan Behnel wrote:
> DasIch, 28.04.2011 20:55:
>> the CPython
>> benchmarks have an extensive set of microbenchmarks in the pybench
>> package
> 
> Try not to care too much about pybench. There is some value in it, but
> some of its microbenchmarks are also tied to CPython's interpreter
> behaviour. For example, the benchmarks for literals can easily be
> considered dead code by other Python implementations so that they may
> end up optimising the benchmarked code away completely, or at least
> partially. That makes a comparison of the results somewhat pointless.

The point of the micro benchmarks in pybench is to be able to compare
them one-by-one, not by looking at the sum of the tests.

If one implementation optimizes away some parts, then the comparison
will show this fact very clearly - and that's the whole point.

Taking the sum of the micro benchmarks only has some meaning
as very rough indicator of improvement. That's why I wrote pybench:
to get a better, more details picture of what's happening,
rather than trying to find some way of measuring "average"
use.

This "average" is very different depending on where you look:
for some applications method calls may be very important,
for others, arithmetic operations, and yet others may have more
need for fast attribute lookup.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Apr 28 2011)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2011-06-20: EuroPython 2011, Florence, Italy               53 days to go

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/