[SciPy-user] Benchmark data

[Arnd Baecker]

Hi,

> > So, I'm not sure how to reproduce what Gerard sees (except numarray's
> > faster arange)
> > which is a little perplexing.  I suppose that's why people criticize
> > benchmarks so much.

Maybe this one can explain it:

> Just one remark on bench.py: it uses time.time().
> So it does not determine the CPU time of a process.
> This could be determined with jiffies
>   from scipy.test.testing import jiffies
> Another option might be timeit.py, see
>   In [4]: import timeit
>   In [5]: timeit?
> (worth reading!)

I just tested bench.py with an updated installation
on my laptop and wanted to compare it with an older
one. The fluctuations between measurements are substantial:

   TOTAL              25.47          18.81          17.27
   TOTAL              22.47          19.03          18.55
   TOTAL              22.59          18.92          15.83
(you will like the last one most, I guess ;-)

If one uses time.clock() - which is one option on Linux,
the timings for the small sizes (4,6, 8?)
are essentially 0.
For 11 one gets pretty stable results:

python bench.py 11
Python 2.3.5 (#2, Sep  4 2005, 22:01:42)
[GCC 3.3.5 (Debian 1:3.3.5-13)]
Optimization flags: -DNDEBUG -g -O3 -Wall -Wstrict-prototypes
CPU info: getNCPUs has_mmx has_sse is_32bit is_Intel is_Pentium
is_PentiumII is_PentiumIII is_i686 is_singleCPU
Numeric-23.8
numarray-1.1.1
scipy-core-0.8.2.1623
benchmark size = 11  (vectors of length 4194304)
label            Numeric       numarray     scipy.base
    1               0.38           0.07           0.33
    2                0.2           0.15           0.24
    3               0.15           0.13           0.23
    4               0.99            0.4           0.41
    5               0.21           0.16           0.16
    6               0.17           0.14           0.22
    7               1.02           0.68           0.42
    8               0.62           0.33           0.33
    9               6.79           5.53           4.76
   10               6.78           6.24           4.79
   11               5.38            4.8            3.9
TOTAL              22.69          18.63          15.79

Presumably we really should use timeit for the smaller
problem sizes ...

Best,  Arnd