[Numpy-discussion] is my numpy installation using custom blas/lapack?

Chris Colbert sccolbert at gmail.com
Sun Jun 7 12:49:48 EDT 2009 

Based on your site.cfg you posted earlier, your install seems to be
working fine. Your site.cfg doesnt have threaded libraries enabled. If
you want to add threads (assuming you built the threaded atlas .so's),
your site.cfg should look like this (assuming the threaded libs are in
the same directory as the non-threaded libs):


##### site.cfg ###########
[DEFAULT]
library_dirs = /usr/local/rich/src/scipy_build/lib:$HOME/usr/galois/lib
include_dirs = /usr/local/rich/src/scipy_build/lib/include:$HOME/usr/galois/include

[blas_opt]
libraries = ptf77blas, ptcblas, atlas

[lapack_opt]
libraries = lapack, ptf77blas, ptcblas, atlas

[amd]
amd_libs = amd

[umfpack]
umfpack_libs = umfpack, gfortran

[fftw]
libraries = fftw3




notice the ptf prefix on the atlas and blas libs. when you did
numpy.show_config() it shows that its only using the single threaded
libs, which is what you told it to do in site.cfg when you built it.

Change your site.cfg rebuild & reinstall and you should be fine

Chris




On Sun, Jun 7, 2009 at 12:11 AM, <llewelr at gmail.com> wrote:
> Hi,
>
> On Jun 6, 2009 3:11pm, Chris Colbert <sccolbert at gmail.com> wrote:
>> it definately found your threaded atlas libraries. How do you know
>>
>> it's numpy is using lapack_lite?
>
> I don't, actually. But it is importing it. With python -v, this is the error
> I get if I don't set LD_LIBRARY_PATH to my scipy_build directory
>
>
> import numpy.linalg.linalg # precompiled from
> /data10/users/rich/usr/galois/lib64/python/numpy/linalg/linalg.pyc
> dlopen("/data10/users/rich/usr/galois/lib64/python/numpy/linalg/lapack_lite.so",
> 2);
> Traceback (most recent call last):
> File "<stdin>", line 1, in <module>
> File "/data10/users/rich/usr/galois//lib64/python/numpy/__init__.py", line
> 130, in <module>
> import add_newdocs
> File "/data10/users/rich/usr/galois//lib64/python/numpy/add_newdocs.py",
> line 9, in <module>
> from lib import add_newdoc
> File "/data10/users/rich/usr/galois//lib64/python/numpy/lib/__init__.py",
> line 13, in <module>
> from polynomial import *
> File "/data10/users/rich/usr/galois//lib64/python/numpy/lib/polynomial.py",
> line 18, in <module>
> from numpy.linalg import eigvals, lstsq
> File "/data10/users/rich/usr/galois//lib64/python/numpy/linalg/__init__.py",
> line 47, in <module>
> from linalg import *
> File "/data10/users/rich/usr/galois//lib64/python/numpy/linalg/linalg.py",
> line 22, in <module>
> from numpy.linalg import lapack_lite
> ImportError: liblapack.so: cannot open shared object file: No such file or
> directory
>>>>
>
>
> Here blas_opt_info seems to be missing ATLAS version.
>
>>>> numpy.show_config()
> atlas_threads_info:
> libraries = ['lapack', 'lapack', 'f77blas', 'cblas', 'atlas']
> library_dirs = ['/usr/local/rich/src/scipy_build/lib']
> language = f77
>
> blas_opt_info:
> libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
> library_dirs = ['/usr/local/rich/src/scipy_build/lib']
> define_macros = [('NO_ATLAS_INFO', 2)]
> language = c
>
> atlas_blas_threads_info:
> libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
> library_dirs = ['/usr/local/rich/src/scipy_build/lib']
> language = c
>
> lapack_opt_info:
> libraries = ['lapack', 'lapack', 'f77blas', 'cblas', 'atlas']
> library_dirs = ['/usr/local/rich/src/scipy_build/lib']
> define_macros = [('NO_ATLAS_INFO', 2)]
> language = f77
>
> lapack_mkl_info:
> NOT AVAILABLE
>
> blas_mkl_info:
> NOT AVAILABLE
>
> mkl_info:
> NOT AVAILABLE
>
>
>>
>>
>>
>> when I do:
>>
>>
>>
>> python
>>
>> >>import numpy
>>
>> >>numpy.show_config()
>>
>> atlas_threads_info:
>>
>>    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
>>
>>    library_dirs = ['/usr/local/lib']
>>
>>    language = f77
>>
>>
>>
>> blas_opt_info:
>>
>>    libraries = ['ptf77blas', 'ptcblas', 'atlas']
>>
>>    library_dirs = ['/usr/local/lib']
>>
>>    define_macros = [('ATLAS_INFO', '"\\"3.8.3\\""')]
>>
>>    language = c
>>
>>
>>
>> atlas_blas_threads_info:
>>
>>    libraries = ['ptf77blas', 'ptcblas', 'atlas']
>>
>>    library_dirs = ['/usr/local/lib']
>>
>>    language = c
>>
>>
>>
>> lapack_opt_info:
>>
>>    libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
>>
>>    library_dirs = ['/usr/local/lib']
>>
>>    define_macros = [('NO_ATLAS_INFO', 2)]
>>
>>    language = f77
>>
>>
>>
>> lapack_mkl_info:
>>
>>  NOT AVAILABLE
>>
>>
>>
>> blas_mkl_info:
>>
>>  NOT AVAILABLE
>>
>>
>>
>> mkl_info:
>>
>>  NOT AVAILABLE
>>
>>
>>
>>
>>
>> also try:
>>
>> >>> a = numpy.random.randn(6000, 6000)
>>
>> >>> numpy.dot(a,a)
>>
>>
>>
>> and make sure all your cpu cores peg at 100%
>>
>>
>>
>
> Unfortunately only one cpu. What does that mean? Threaded libraries not
> used?
>
> from top:
>
> Cpu0 :100.0%us, 0.0%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu1 : 0.0%us, 0.2%sy, 0.0%ni, 99.4%id, 0.0%wa, 0.2%hi, 0.2%si, 0.0%st
> Cpu2 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu3 : 0.0%us, 0.0%sy, 0.0%ni,100.0%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
>
> Thanks much for the help.
>
> Rich
>
>>
>>
>> On Sat, Jun 6, 2009 at 3:35 PM, llewelr at gmail.com> wrote:
>>
>> > Oops. Thanks, that makes more sense:
>>
>> >
>>
>> > http://pastebin.com/m7067709b
>>
>> >
>>
>> > On Jun 6, 2009 12:15pm, Chris Colbert sccolbert at gmail.com> wrote:
>>
>> >> i need the full link to pastebin.com in order to view your post.
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >> It will be something like http://pastebin.com/m6b09f05c
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >> chris
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >>
>>
>> >> On Sat, Jun 6, 2009 at 2:32 PM, Richard Llewellynllewelr at gmail.com>
>> >> wrote:
>>
>> >>
>>
>> >> > I posted the setup.py build output to pastebin.com, though missed the
>>
>> >>
>>
>> >> > uninteresting stderr (forgot tcsh command to redirect both).
>>
>> >>
>>
>> >> > Also, used setup.py build --fcompiler=gnu95.
>>
>> >>
>>
>> >> >
>>
>> >>
>>
>> >> >
>>
>> >>
>>
>> >> > To be clear, I am not certain that my ATLAS libraries are not found.
>> >> > But
>>
>> >>
>>
>> >> > during the build starting at line 95 (pastebin.com) I see a
>> >> > compilation
>>
>> >>
>>
>> >> > failure, and then NO_ATLAS_INFO, 2.
>>
>> >>
>>
>> >> >
>>
>> >>
>>
>> >> > I don't think I can use ldconfig without root, but have set
>>
>> >> > LD_LIBRARY_PATH
>>
>> >>
>>
>> >> > to point to the scipy_build/lib until I put them somewhere else.
>>
>> >>
>>
>> >> >
>>
>> >>
>>
>> >> > importing numpy works, though lapack_lite is also imported. I wonder
>> >> > if
>>
>> >> > this
>>
>> >>
>>
>> >> > is normal even if my ATLAS was used.
>>
>> >>
>>
>> >> >
>>
>> >>
>>
>> >> > Thanks,
>>
>> >>
>>
>> >> > Rich
>>
>> >>
>>
>> >> >
>>
>> >>
>>
>> >> > On Sat, Jun 6, 2009 at 10:46 AM, Chris Colbert sccolbert at gmail.com>
>>
>> >> > wrote:
>>
>> >>
>>
>> >> >>
>>
>> >>
>>
>> >> >> and where exactly are you seeing atlas not found? during the build
>>
>> >>
>>
>> >> >> process, are when import numpy in python?
>>
>> >>
>>
>> >> >>
>>
>> >>
>>
>> >> >> if its the latter, you need to add a .conf file  in
>> >> >> /etc/ld.so.conf.d/
>>
>> >>
>>
>> >> >>  with the line /usr/local/rich/src/scipy_build/lib  and then run
>> >> >>  sudo
>>
>> >>
>>
>> >> >> ldconfig
>>
>> >>
>>
>> >> >>
>>
>> >>
>>
>> >> >> Chris
>>
>> >>
>>
>> >> >>
>>
>> >>
>>
>> >> >>
>>
>> >>
>>
>> >> >> On Sat, Jun 6, 2009 at 1:42 PM, Chris Colbertsccolbert at gmail.com>
>>
>> >> >> wrote:
>>
>> >>
>>
>> >> >> > can you run this and post the build.log to pastebin.com:
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> > assuming your numpy build directory is /home/numpy-1.3.0:
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> > cd /home/numpy-1.3.0
>>
>> >>
>>
>> >> >> > rm -rf build
>>
>> >>
>>
>> >> >> > python setup.py build &&> build.log
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> > Chris
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> > On Sat, Jun 6, 2009 at 1:37 PM, Richard
>> >> >> > Llewellynllewelr at gmail.com>
>>
>> >>
>>
>> >> >> > wrote:
>>
>> >>
>>
>> >> >> >> Hi Chris,
>>
>> >>
>>
>> >> >> >>  thanks much for posting those installation instructions.  Seems
>>
>> >>
>>
>> >> >> >> similar to
>>
>> >>
>>
>> >> >> >> what I pieced together.
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> I gather ATLAS not found.  Oops, drank that beer too early.
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> I copied Atlas libs to /usr/local/rich/src/scipy_build/lib.
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> This is my site.cfg.  Out of desperation I tried
>> >> >> >> search_static_first
>>
>> >> >> >> =
>>
>> >>
>>
>> >> >> >> 1,
>>
>> >>
>>
>> >> >> >> but probably of no use.
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> [DEFAULT]
>>
>> >>
>>
>> >> >> >> library_dirs =
>>
>> >> >> >> /usr/local/rich/src/scipy_build/lib:$HOME/usr/galois/lib
>>
>> >>
>>
>> >> >> >> include_dirs =
>>
>> >>
>>
>> >> >> >>
>> >> >> >> /usr/local/rich/src/scipy_build/lib/include:$HOME/usr/galois/include
>>
>> >>
>>
>> >> >> >> search_static_first = 1
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> [blas_opt]
>>
>> >>
>>
>> >> >> >> libraries = f77blas, cblas, atlas
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> [lapack_opt]
>>
>> >>
>>
>> >> >> >> libraries = lapack, f77blas, cblas, atlas
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> [amd]
>>
>> >>
>>
>> >> >> >> amd_libs = amd
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> [umfpack]
>>
>> >>
>>
>> >> >> >> umfpack_libs = umfpack, gfortran
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> [fftw]
>>
>> >>
>>
>> >> >> >> libraries = fftw3
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> Rich
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> On Sat, Jun 6, 2009 at 10:25 AM, Chris Colbert
>> >> >> >> sccolbert at gmail.com>
>>
>> >>
>>
>> >> >> >> wrote:
>>
>> >>
>>
>> >> >> >>>
>>
>> >>
>>
>> >> >> >>> when you build numpy, did you use site.cfg to tell it where to
>> >> >> >>> find
>>
>> >>
>>
>> >> >> >>> your atlas libs?
>>
>> >>
>>
>> >> >> >>>
>>
>> >>
>>
>> >> >> >>> On Sat, Jun 6, 2009 at 1:02 PM, Richard
>> >> >> >>> Llewellynllewelr at gmail.com>
>>
>> >>
>>
>> >> >> >>> wrote:
>>
>> >>
>>
>> >> >> >>> > Hello,
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > I've managed a build of lapack and atlas on Fedora 10 on a
>> >> >> >>> > quad
>>
>> >>
>>
>> >> >> >>> > core,
>>
>> >>
>>
>> >> >> >>> > 64,
>>
>> >>
>>
>> >> >> >>> > and now (...) have a numpy I can import that runs tests ok. :]
>>
>> >> >> >>> > I
>>
>> >>
>>
>> >> >> >>> > am
>>
>> >>
>>
>> >> >> >>> > puzzled, however, that numpy builds and imports lapack_lite.
>>
>> >> >> >>> > Does
>>
>> >>
>>
>> >> >> >>> > this
>>
>> >>
>>
>> >> >> >>> > mean
>>
>> >>
>>
>> >> >> >>> > I have a problem with the build(s)?
>>
>> >>
>>
>> >> >> >>> > Upon building numpy, I see the troubling output:
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > ########################
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > C compiler: gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g
>>
>> >> >> >>> > -pipe
>>
>> >>
>>
>> >> >> >>> > -Wall
>>
>> >>
>>
>> >> >> >>> > -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protecto
>>
>> >>
>>
>> >> >> >>> > r --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE
>>
>> >> >> >>> > -fPIC
>>
>> >>
>>
>> >> >> >>> > -fPIC
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > compile options: '-c'
>>
>> >>
>>
>> >> >> >>> > gcc: _configtest.c
>>
>> >>
>>
>> >> >> >>> > gcc -pthread _configtest.o
>> >> >> >>> > -L/usr/local/rich/src/scipy_build/lib
>>
>> >>
>>
>> >> >> >>> > -llapack
>>
>> >>
>>
>> >> >> >>> > -lptf77blas -lptcblas -latlas -o _configtest
>>
>> >>
>>
>> >> >> >>> > /usr/bin/ld: _configtest: hidden symbol `__powidf2' in
>>
>> >>
>>
>> >> >> >>> > /usr/lib/gcc/x86_64-redhat-linux/4.3.2/libgcc.a(_powidf2.o) is
>>
>> >>
>>
>> >> >> >>> > reference
>>
>> >>
>>
>> >> >> >>> > d by DSO
>>
>> >>
>>
>> >> >> >>> > /usr/bin/ld: final link failed: Nonrepresentable section on
>>
>> >> >> >>> > output
>>
>> >>
>>
>> >> >> >>> > collect2: ld returned 1 exit status
>>
>> >>
>>
>> >> >> >>> > /usr/bin/ld: _configtest: hidden symbol `__powidf2' in
>>
>> >>
>>
>> >> >> >>> > /usr/lib/gcc/x86_64-redhat-linux/4.3.2/libgcc.a(_powidf2.o) is
>>
>> >>
>>
>> >> >> >>> > reference
>>
>> >>
>>
>> >> >> >>> > d by DSO
>>
>> >>
>>
>> >> >> >>> > /usr/bin/ld: final link failed: Nonrepresentable section on
>>
>> >> >> >>> > output
>>
>> >>
>>
>> >> >> >>> > collect2: ld returned 1 exit status
>>
>> >>
>>
>> >> >> >>> > failure.
>>
>> >>
>>
>> >> >> >>> > removing: _configtest.c _configtest.o
>>
>> >>
>>
>> >> >> >>> > Status: 255
>>
>> >>
>>
>> >> >> >>> > Output:
>>
>> >>
>>
>> >> >> >>> >   FOUND:
>>
>> >>
>>
>> >> >> >>> >     libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
>>
>> >>
>>
>> >> >> >>> >     library_dirs = ['/usr/local/rich/src/scipy_build/lib']
>>
>> >>
>>
>> >> >> >>> >     language = f77
>>
>> >>
>>
>> >> >> >>> >     define_macros = [('NO_ATLAS_INFO', 2)]
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > ##########################
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > I don't have root on this machine, but could pester admins for
>>
>> >>
>>
>> >> >> >>> > eventual
>>
>> >>
>>
>> >> >> >>> > temporary access.
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > Thanks much for any help,
>>
>> >>
>>
>> >> >> >>> > Rich
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> > _______________________________________________
>>
>> >>
>>
>> >> >> >>> > Numpy-discussion mailing list
>>
>> >>
>>
>> >> >> >>> > Numpy-discussion at scipy.org
>>
>> >>
>>
>> >> >> >>> > http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> >
>>
>> >>
>>
>> >> >> >>> _______________________________________________
>>
>> >>
>>
>> >> >> >>> Numpy-discussion mailing list
>>
>> >>
>>
>> >> >> >>> Numpy-discussion at scipy.org
>>
>> >>
>>
>> >> >> >>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >> _______________________________________________
>>
>> >>
>>
>> >> >> >> Numpy-discussion mailing list
>>
>> >>
>>
>> >> >> >> Numpy-discussion at scipy.org
>>
>> >>
>>
>> >> >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >>
>>
>> >>
>>
>> >> >> >
>>
>> >>
>>
>> >> >> _______________________________________________
>>
>> >>
>>
>> >> >> Numpy-discussion mailing list
>>
>> >>
>>
>> >> >> Numpy-discussion at scipy.org
>>
>> >>
>>
>> >> >> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> >>
>>%
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>

More information about the NumPy-Discussion mailing list