[Speed] CPython Benchmark Suite usinf my perf module, GitHub, etc.

[Zachary Ware]

On Fri, Jul 29, 2016 at 12:03 PM, Brett Cannon <brett at python.org> wrote:
> On Thu, 28 Jul 2016 at 10:25 Victor Stinner <victor.stinner at gmail.com>
> wrote:
>> * Should we remove vendor copies of libraries and work with virtual
>> environments? Not all libraries are available on PyPI :-/ See the
>> requirements.txt file and TODO.
>
> If they are not on PyPI then we should just drop the benchmark. And I say we
> do use virtual environments to keep the repo size down.

I think rather than using virtual environments which aren't truly
supported by <3.3 anyway, we should instead make use of pip's
--target, --root, and/or --prefix flags (whatever combination it
takes, I haven't looked into it deeply) to install the packages into a
particular dir which is then added to each benchmarked interpreter's
PYTHONPATH.  This way, we're sure that each interpreter is running
exactly the same code.

Either way, I'm for not vendoring libraries.  If the library
disappears from PyPI, it's probably not an important workload anymore
anyway.

>> Reminder: My final goal is to merge again all benchmarks suites
>> (CPython, PyPy, Pyston, Pyjion, ...) into one unique project!
>
> I hope this happens!

Me too!  I'd also like to get the benchmark runner for
speed.python.org set up to build and benchmark as many interpreters as
possible.

-- 
Zach