[Python-Dev] GSoC: speed.python.org

[Antoine Pitrou]

On Mon, 21 Mar 2011 14:30:45 -0500
skip at pobox.com wrote:
> 
>     >> Is there any value in them as regression tests (maybe with more
>     >> elaborate inputs and/or longer runtimes)?
> 
>     Antoine> You mean to check behaviour or to check for performance
>     Antoine> regressions?
> 
> Both.  Semantic regressions, and secondarily, performance regressions.
> I can understand the need to stabilize the code and inputs for measuring
> performance.  As bad a benchmark as pystone is, one of its saving graces is
> that it hasn't changed in so long.

I think that semantic regressions would be better checked by running
the test suites of these third-party libraries, not benchmarks.
Ideally a separate suite of buildbots or builders could do that. We had
the community buildbots but it seems they ran out of steam.

Regards

Antoine.