[Numpy-discussion] Unable to building numpy with openblas using bento or distutils
Skipper Seabold
jsseabold at gmail.com
Sat Mar 23 14:19:42 EDT 2013
Some help on this would be greatly appreciated. It's been recommended to
use OpenBlas over ATLAS, so I've been trying to build numpy with openblas
and have run into a few problems.
1) Build fails using bento master and waf 1.7.9, see below.
2) Distutils doesn't seem to be able to find lapack as part of atlas. I
tried to skip a site.cfg and define environmental variables. No idea what I
missed.
I followed instructions found scattered over the internet and only
understand vaguely the issues. Maybe someone can help. I'd be happy to
update the wiki with any answers.
To truly support OpenBlas, is it maybe necessary to make some additions to
numpy/distutils/system_info.py?
Thanks for having a look,
Skipper
Install OpenBlas
-----------------------------
git clone git://github.com/xianyi/OpenBLAS
cd OpenBlas
Edit c_check to look for libpthreads in the right place (Kubuntu 12.10)
|4 $ git diff c_check
```
diff --git a/c_check b/c_check
index 4d82237..de0fd33 100644
--- a/c_check
+++ b/c_check
@@ -241,7 +241,7 @@ print CONFFILE "#define FUNDERSCORE\t$need_fu\n" if
$need_fu
if ($os eq "LINUX") {
- @pthread = split(/\s+/, `nm /lib/libpthread.so* | grep
_pthread_create`);
+ @pthread = split(/\s+/, `nm /lib/x86_64-linux-gnu/libpthread.so* |
grep _pthread_create`);
if ($pthread[2] ne "") {
print CONFFILE "#define PTHREAD_CREATE_FUNC $pthread[2]\n";
```
make fc=gfortran
make PREFIX=~/.local install
Everything looks ok, so far.
Install NumPy
---------------------------
Using numpy master
I tried to use bento master and waf 1.7.9, following instructions from
David's blog
bentomaker configure --prefix=/home/skipper/.local
--with-blas-lapack-libdir=/home/skipper/.local/lib
--blas-lapack-type=openblas ..
bentomaker build -j4
```
<snip>
[101/104] cshlib: build/numpy/core/src/umath/umath_tests.c.11.o ->
build/numpy/core/umath_tests.so
/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
/usr/bin/ld: numpy/core/lib/libnpymath.a(halffloat.c.12.o): relocation
R_X86_64_PC32 against symbol `npy_halfbits_to_floatbits' can not be used
when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
```
No idea, so, let's try distutils
export LAPACK=~/.local/lib/libopenblas.a
export BLAS=~/.local/lib/libopenblas.a
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/.local/lib/
echo $LD_LIBRARY_PATH
```
:/usr/local/lib64/R/bin:/home/skipper/.local/lib/
```
This step seems to be necessary?
python setup.py config
```
Running from numpy source directory.
non-existing path in 'numpy/distutils': 'site.cfg'
F2PY Version 2
numpy/core/setup_common.py:88: MismatchCAPIWarning: API mismatch detected,
the C API version numbers have to be updated. Current C api version is 8,
with checksum f4362353e2d72f889fda0128aa015037, but recorded checksum for C
API version 8 in codegen_dir/cversions.txt is
17321775fc884de0b1eda478cd61c74b. If functions were added in the C API, you
have to update C_API_VERSION in numpy/core/setup_common.py.
MismatchCAPIWarning)
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
atlas_blas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
atlas_blas_info:
libraries f77blas,cblas,atlas not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1501:
UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
blas_info:
Replacing _lib_names[0]=='blas' with 'openblas'
Replacing _lib_names[0]=='openblas' with 'openblas'
FOUND:
libraries = ['openblas']
library_dirs = ['/home/skipper/.local/lib']
language = f77
FOUND:
libraries = ['openblas']
library_dirs = ['/home/skipper/.local/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77
non-existing path in 'numpy/lib': 'benchmarks'
lapack_opt_info:
lapack_mkl_info:
mkl_info:
libraries mkl,vml,guide not found in ['/usr/local/lib64',
'/usr/local/lib', '/usr/lib64', '/usr/lib', '/usr/lib/x86_64-linux-gnu']
NOT AVAILABLE
NOT AVAILABLE
atlas_threads_info:
Setting PTATLAS=ATLAS
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib64
libraries lapack_atlas not found in /usr/local/lib64
libraries ptf77blas,ptcblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib64
libraries lapack_atlas not found in /usr/lib64
libraries ptf77blas,ptcblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
libraries ptf77blas,ptcblas,atlas not found in /usr/lib/x86_64-linux-gnu
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
numpy.distutils.system_info.atlas_threads_info
NOT AVAILABLE
atlas_info:
libraries f77blas,cblas,atlas not found in /usr/local/lib64
libraries lapack_atlas not found in /usr/local/lib64
libraries f77blas,cblas,atlas not found in /usr/local/lib
libraries lapack_atlas not found in /usr/local/lib
libraries f77blas,cblas,atlas not found in /usr/lib64
libraries lapack_atlas not found in /usr/lib64
libraries f77blas,cblas,atlas not found in /usr/lib
libraries lapack_atlas not found in /usr/lib
libraries f77blas,cblas,atlas not found in /usr/lib/x86_64-linux-gnu
libraries lapack_atlas not found in /usr/lib/x86_64-linux-gnu
numpy.distutils.system_info.atlas_info
NOT AVAILABLE
/home/skipper/src/numpy-skipper/numpy/distutils/system_info.py:1415:
UserWarning:
Atlas (http://math-atlas.sourceforge.net/) libraries not found.
Directories to search for the libraries can be specified in the
numpy/distutils/site.cfg file (section [atlas]) or by setting
the ATLAS environment variable.
warnings.warn(AtlasNotFoundError.__doc__)
lapack_info:
Replacing _lib_names[0]=='lapack' with 'openblas'
Replacing _lib_names[0]=='openblas' with 'openblas'
FOUND:
libraries = ['openblas']
library_dirs = ['/home/skipper/.local/lib']
language = f77
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/home/skipper/.local/lib']
define_macros = [('NO_ATLAS_INFO', 1)]
language = f77
running config
```
python setup.py build &> build.log
Build log is here. Obviously it didn't go well, but I don't see anything to
indicate problems. Sometimes I am able to get _dotblas.so built, though I
don't know what causes it. This time I wasn't.
https://gist.github.com/jseabold/7054ba9d85eae09eb402#file-numpy_build-log
sudo python setup.py install &> install.log
https://gist.github.com/jseabold/a0f5638b65d44aeff598#file-numpy_install-log
>>> import numpy as np
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/numpy/__init__.py", line
138, in <module>
import add_newdocs
File "/usr/local/lib/python2.7/dist-packages/numpy/add_newdocs.py", line
13, in <module>
from numpy.lib import add_newdoc
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/__init__.py", line
15, in <module>
from polynomial import *
File "/usr/local/lib/python2.7/dist-packages/numpy/lib/polynomial.py",
line 19, in <module>
from numpy.linalg import eigvals, lstsq, inv
File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/__init__.py",
line 50, in <module>
from linalg import *
File "/usr/local/lib/python2.7/dist-packages/numpy/linalg/linalg.py",
line 25, in <module>
from numpy.linalg import lapack_lite
ImportError: libopenblas.so.0: cannot open shared object file: No such file
or directory
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20130323/d58b5817/attachment.html>
More information about the NumPy-Discussion
mailing list