[Numpy-discussion] Matlab/Numeric/numarray benchmarks

[Todd Miller]

On Wed, 2005-01-05 at 19:23, Simon Burton wrote:
> Todd Miller wrote:
> 
> >Hi Simon,
> >
> >I found a benchmark bug which explains the performance difference in +. 
> >Here are my times with the modified benchmark (Python-2.4 gcc version
> >3.2.2 20030222 (Red Hat Linux 3.2.2-5) on 1.7 GHz P-IV w/ 2G):
> >
> >numarray + : 0.0540893316269
> >numarray matrixmultiply : 16.9448821545
> >numarray eigenvalues : 9.67254910469
> >
> >Numeric + : 0.0653991508484
> >Numeric matrixmultiply : 33.0565470934
> >Numeric eigenvalues : 9.44225819111
> >
> >So, for large arrays with a simple to install / built-in linear algebra
> >system,  numarray is doing just fine.  
> >
> >Looking at your results,  I think you may have been comparing numarray
> >built using a built-in blas_lite versus Numeric using ATLAS.  There,  I
> >think numarray *is* behind but that is fixable with some effort.  The
> >key is porting and integrating the Numeric dotblas package with
> >numarray.  I've been looking at that some today...  err, yesterday,
> >apparently I forgot to hit "send".
> >  
> >
> 
> Wow, those results look great, Todd.
>
> I have double checked my install. However, the numarray multiply is 
> still x10 slower.
> 
> This is set in addons:
>  lapack_libs = ['lapack', 'f77blas', 'cblas', 'atlas', 'blas']
> 
> and, at runtime, python has loaded:
> 40771000-40cb6000 r-xp 00000000 00:0c 783245     
> /usr/lib/atlas/liblapack.so.3.0
> 40cb6000-40cb9000 rw-p 00545000 00:0c 783245     
> /usr/lib/atlas/liblapack.so.3.0
> 40cb9000-40dbd000 rw-p 00000000 00:00 0
> 40dbd000-40dd7000 r-xp 00000000 00:0c 783242     /usr/lib/libf77blas.so.3.0
> 40dd7000-40dd8000 rw-p 00019000 00:0c 783242     /usr/lib/libf77blas.so.3.0
> 40dd8000-40df7000 r-xp 00000000 00:0c 783241     /usr/lib/libcblas.so.3.0
> 40df7000-40df8000 rw-p 0001e000 00:0c 783241     /usr/lib/libcblas.so.3.0
> 40df8000-4110b000 r-xp 00000000 00:0c 783240     /usr/lib/libatlas.so.3.0
> 4110b000-4110f000 rw-p 00312000 00:0c 783240     /usr/lib/libatlas.so.3.0
> 4110f000-41454000 r-xp 00000000 00:0c 783244     
> /usr/lib/atlas/libblas.so.3.0
> 41454000-41458000 rw-p 00345000 00:0c 783244     
> /usr/lib/atlas/libblas.so.3.0
> 41458000-41472000 r-xp 00000000 00:0c 17227      /usr/lib/libg2c.so.0.0.0
> 41472000-41473000 rw-p 0001a000 00:0c 17227      /usr/lib/libg2c.so.0.0.0
> ( as well as lapack_lite2.so )
> 
> So I assumed that since the eigenvalues came in fast ATLAS was alive and 
> well.
> Also, the above libs are exactly what Numeric uses:
> 4033b000-40880000 r-xp 00000000 00:0c 783245     
> /usr/lib/atlas/liblapack.so.3.0
> 40880000-40883000 rw-p 00545000 00:0c 783245     
> /usr/lib/atlas/liblapack.so.3.0
> 40883000-40987000 rw-p 00000000 00:00 0
> 40987000-409a6000 r-xp 00000000 00:0c 783241     /usr/lib/libcblas.so.3.0
> 409a6000-409a7000 rw-p 0001e000 00:0c 783241     /usr/lib/libcblas.so.3.0
> 409a7000-409c1000 r-xp 00000000 00:0c 783242     /usr/lib/libf77blas.so.3.0
> 409c1000-409c2000 rw-p 00019000 00:0c 783242     /usr/lib/libf77blas.so.3.0
> 409c2000-40cd5000 r-xp 00000000 00:0c 783240     /usr/lib/libatlas.so.3.0
> 40cd5000-40cd9000 rw-p 00312000 00:0c 783240     /usr/lib/libatlas.so.3.0
> 40cd9000-40cf3000 r-xp 00000000 00:0c 17227      /usr/lib/libg2c.so.0.0.0
> 40cf3000-40cf4000 rw-p 0001a000 00:0c 17227      /usr/lib/libg2c.so.0.0.0
> 40cf4000-40cf7000 rw-p 00000000 00:00 0
> 40cf7000-4103c000 r-xp 00000000 00:0c 783244     
> /usr/lib/atlas/libblas.so.3.0
> 4103c000-41040000 rw-p 00345000 00:0c 783244     
> /usr/lib/atlas/libblas.so.3.0
> 
> 
> Any ideas ? 

<slow drawling prison warden voice from Cool Hand Luke>
What we've got here is... a'falya to communicate.
</slow drawling prison warden voice from Cool Hand Luke>

numarray's dot/matrixmultiply() and innerproduct() have never been
implemented in terms of a BLAS.  *That's* the problem.  Numeric has the
"dotblas" extension which augments the the built-in versions of these
functions with ones that are souped up using external libraries.  

I ported dotblas yesterday and checked it into numarray CVS yesterday
afternoon.  I fixed the last doctest artifact and re-arranged a little
this morning.

If you're working from CVS you should be able to see the new performance
optimization now by doing an update and building/linking against the
right libraries.  To do that I:

setenv USE_LAPACK 1
setenv LINALG_LIB <where my libraries are>
setenv LINALG_INCLUDE <where my includes are>

Have a look.  I think numarray is slightly ahead now.

Todd