LinearAlgebra incredibly slow for eigenvalue problems

[David M. Cooke]

"drife" <daranrife at yahoo.com> writes:

> Hello,
>
> I need to calculate the eigenvectors and eigenvalues for a 3600 X 3600
> covariance matrix.
>
> The LinearAlgebra package in Python is incredibly slow to perform the
> above calculations (about 1.5 hours). This in spite of the fact that
> I have installed Numeric with the full ATLAS and LAPACK libraries.
>
> Also note that my computer has dual Pentium IV (3.1 GHz) processors
> with 2Gb ram.
>
> Every Web discussion I have seen about such issues indicates that
> one can expect huge speed ups if one compiles and installs Numeric
> linked against the ATLAS and LAPACK libraries.

Are you *sure* that Numeric is linked against these?

> Even more perplexing is that the same calculation takes a mere 7 min
> in Matlab V6.5. Matlab uses both ATLAS and LAPACK.
>
> Moreover, the above calculation takes the same amount of time for
> Numeric to complete with --and-- without ATLAS and PACK. I am certain
> that I have done the install correctly.

This is good evidence that Numeric *isn't* linked to them.

If you're on a Linux system, you can check with ldd:
cookedm at arbutus$ ldd /usr/lib/python2.3/site-packages/Numeric/lapack_lite.so 
        liblapack.so.3 => /usr/lib/atlas/liblapack.so.3 (0x0000002a95677000)
        libblas.so.3 => /usr/lib/atlas/libblas.so.3 (0x0000002a95e55000)
        libg2c.so.0 => /usr/lib/libg2c.so.0 (0x0000002a96721000)
        libpthread.so.0 => /lib/libpthread.so.0 (0x0000002a96842000)
        libc.so.6 => /lib/libc.so.6 (0x0000002a96957000)
        libm.so.6 => /lib/libm.so.6 (0x0000002a96b96000)
        /lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 (0x000000552aaaa000)

You can see that lapack and blas (the Atlas versions) are linked to
the lapack_lite.so.

To install Numeric using Lapack:
- remove the build/ directory in your Numeric sources, so you don't
  any old binaries
- edit setup.py and follow the comments on using Lapack (you need to
  comment out a few lines, and set some directories)
  Also set use_dotblas to 1.
- do the 'python setup.py build', 'python setup.py install' dance.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca