unbiased benchmark

[Albert Hopkins]

On Thu, 2009-03-12 at 13:25 -0700, Chris Rebert wrote:
> On Thu, Mar 12, 2009 at 1:07 PM, Sam Ettessoc <saminsf at gmail.com> wrote:
> > I would like to share a benchmark I did. The computer used was a
> > 2160MHz Intel Core Duo w/ 2000MB of 667MHz DDR2 SDRAM running MAC OS
> > 10.5.6 and a lots of software running (a typical developer
> > workstation).
> >
> > Python benchmark:
> > HAMBURGUESA:benchmark sam$ echo 1+1 > bench.py
> > HAMBURGUESA:benchmark sam$ time python bench.py
> > real    0m0.064s
> > user    0m0.049s
> > sys     0m0.013s
> >
> > Ruby benchmark:
> > HAMBURGUESA:benchmark sam$ echo 1+1 > bench.rb
> > HAMBURGUESA:benchmark sam$ time ruby bench.rb
> > real    0m0.006s
> > user    0m0.003s
> > sys     0m0.003s
> >
> > Can you believe it? Ruby is 10 times faster than Python.
> 
> I submit that you are effectively just comparing start-up times:
> 
> $ time ruby < /dev/null
> 
> real	0m0.006s
> user	0m0.003s
> sys	0m0.003s
> 
> $ time python < /dev/null
> 
> real	0m0.020s
> user	0m0.011s
> sys	0m0.009s
> 
> Since Python includes a full interactive interpreter REPL, whereas
> Ruby doesn't, you're comparing apples to oranges. A more fair
> comparison would be to compare python and irb:
> 
> $ time irb </dev/null
> 
> real	0m0.024s
> user	0m0.017s
> sys	0m0.006s
> 

<devil's advocate>
In all fairness, if python isn't being run from a terminal then there
should be no need to initiate a "full interactive interpreter".
</devil's advocate>

But yeah, it's not really a benchmark as it merely shows start-up times.