[Speed] New benchmark suite for Python

[Maciej Fijalkowski]

On Fri, Aug 19, 2016 at 1:20 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 19 August 2016 at 01:55, Victor Stinner <victor.stinner at gmail.com> wrote:
>> 2016-08-18 8:48 GMT+02:00 Armin Rigo <arigo at tunes.org>:
>>> Indeed, bzr cannot be installed on PyPy because it uses Cython in a
>>> strange way: it declares and directly pokes inside PyListObjects from
>>> a .pyx file.  But note that bzr (seems to) have systematically a pure
>>> Python version of all its .pyx files. (...)
>>
>> bazar is only used for a "startup" benchmark. I don't think that such
>> benchmark is very interesting... I would prefer to see a benchmark on
>> a less dummy operation on the repository than displaying the help...
>
> Simple commands like displaying help messages are where interpreter
> startup time dominates the end user experience for applications
> written in Python, though. For example, improvements to import system
> performance tend to mostly show up there - for longer running
> benchmarks, changes in startup time tend to get swamped by the actual
> runtime speed, while the baseline "python -c 'pass'" mainly varies
> based on how many modules we're implicitly importing at startup rather
> than how well the import system is performing .
>
> Cheers,
> Nick.

I would still argue that displaying help is not a very good benchmark :-)