[Python-Dev] Benchmarking Python 3.3 against Python 2.7 (wide build)

[Antoine Pitrou]

On Sun, 30 Sep 2012 21:49:20 -0400
Brett Cannon <brett at python.org> wrote:
> > Note that Mako can use the Markupsafe library for faster operation.
> > This will skew the result if one of your Pythons has Markupsafe
> > installed and the other does not.
> >
> 
> Should probably have the benchmark print out a warning when markupsafe is
> used. Turns out I have it installed in my user directory for Python 2.7 so
> that probably came into play.
> 
> 
> >
> > Perhaps the benchmark runner should launch its subtests in a controlled
> > environment to avoid such issues?
> >
> 
> If we had venv in Python 2.7 that might be easy to do, but otherwise is
> there an easy way without having to try to pull in virtualenv or something
> crazy like a chroot or something?

The mako benchmark could manually exclude markupsafe from sys.modules.
That only addresses that specific benchmark, though.

Regards

Antoine.