[Python-Dev] [GSoC] Developing a benchmark suite (for Python 3.x)

[Anthony Scopatz]

On Thu, Apr 7, 2011 at 6:11 PM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

> On 07/04/2011 22:41, Antoine Pitrou wrote:
>
>> On Thu, 07 Apr 2011 17:32:24 -0400
>> Tres Seaver<tseaver at palladion.com>  wrote:
>>
>>> Right now, we are talking about building "speed.python.org" to test
>>>> the speed of python interpreters, over time, and alongside one another
>>>> - cython *is not* an interpreter.
>>>>
>>>> Cython is out of scope for this.
>>>>
>>> Why is it out of scope to use the benchmarks and test harness to answer
>>> questions like "can we use Cython to provide optional optimizations for
>>> the stdlib"?  I can certainly see value in havng an objective way to
>>> compare the macro benchmark performance of a Cython-optimized CPython
>>> vs. a vanilla CPython, as well as vs. PyPY, Jython, or IronPython.
>>>
>> Agreed. Assuming someone wants to take care of the Cython side of
>> things, I don't think there's any reason to exclude it under the
>> dubious reason that it's "not an interpreter".
>> (would you exclude Psyco, if it was still alive?)
>>
>>
> Well, sure - but within the scope of a GSOC project limiting it to "core
> python" seems like a more realistic goal.
>
> Adding cython later shouldn't be an issue if someone is willing to do the
> work.


Jesse, I understand that we are talking about the benchmarks on
speed.pypy.org.  The current suite, and correct me if I
am wrong, is completely written in pure python so that any of the
'interpreters' may run them.

My point, which I stand by, was that during the initial phase (where
benchmarks are defined) that the Cython crowd
should have a voice.  This should have an enriching effect on the whole
benchmarking task since they have
thought about this issue in a way that is largely orthogonal to the methods
PyPy developed.  I think it
would be a mistake to leave Cython out of the scoping study.

I actually agree with Micheal.  I think the onus of getting the benchmarks
working on every platform is the
onus of that interpreter's community.

The benchmarking framework that is being developed as part of GSoC should be
agile enough to add and
drop projects over time and be able to make certain tests as 'known
failures', etc.

I don't think I am asking anything unreasonable here.  Especially, since at
the end of the day the purview of
projects like PyPy and Cython ("Make Python Faster") is the same.

Be Well
Anthony


>
>
> All the best,
>
> Michael Foord
>
>  Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>>
>
>
> --
> http://www.voidspace.org.uk/
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing http://www.sqlite.org/different.html
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/scopatz%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20110407/fac291a8/attachment-0001.html>