[Python-Dev] performance testing recommendations in devguide

[M.-A. Lemburg]

On 29.05.2013 21:19, Antoine Pitrou wrote:
> 
> Hi,
> 
> On Wed, 29 May 2013 21:59:21 +0300
> Carlos Nepomuceno <carlosnepomuceno at outlook.com> wrote:
>>
>> [1] "pybench - run the standard Python PyBench benchmark suite. This is considered
>> an unreliable, unrepresentative benchmark; do not base decisions
>> off it. It is included only for completeness."
> 
> "unrepresentative" is the main criticism against pybench. PyBench is a
> suite of micro-benchmarks (almost nano-benchmarks, actually :-)) that
> don't try to simulate any real-world situation.
> 
> PyBench may also be unreliable, because its tests are so static that
> they could be optimized away by a clever enough (JIT) compiler.

Correct.

pybench was written to test and verify CPython interpreter
optimizations and also to detect changes which resulted
in performance degradation of very basic operations such as
attribute lookups, method calls, simple integer math, etc.

It was never meant to be representative of anything :-)

At the time, we only had pystone as "benchmark" and things
like high precision timers were not yet readily available
as they are now.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 29 2013)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2013-07-01: EuroPython 2013, Florence, Italy ...           33 days to go

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/