[SciPy-dev] newcore on 64 bit systems
Arnd Baecker
arnd.baecker at web.de
Wed Nov 9 08:13:56 EST 2005
Hi Nils,
On Wed, 9 Nov 2005, Nils Wagner wrote:
> I have also a problem with python setup.py install in newcore
>
> gcc options: '-pthread -fno-strict-aliasing -DNDEBUG -O2
> -fmessage-length=0 -Wall -g -fPIC'
> compile options: '-DNO_ATLAS_INFO=1 -Iscipy/base/include
> -Ibuild/src/scipy/base -Iscipy/base/src -I/usr/include/python2.4 -c'
> /usr/bin/g77 -shared
> build/temp.linux-x86_64-2.4/scipy/corelib/lapack_lite/lapack_litemodule.o
> -L/usr/local/builds/src/blas -Lbuild/temp.linux-x86_64-2.4 -lflapack_src
> -lfblas -lg2c -o build/lib.linux-x86_64-2.4/scipy/lib/lapack_lite.so
> /usr/lib64/gcc-lib/x86_64-suse-linux/3.3.5/../../../../x86_64-suse-linux/bin/ld:
> /usr/local/builds/src/blas/libfblas.a(dgemm.o): relocation R_X86_64_32
> against `a local symbol' can not be used when making a shared object;
> recompile with -fPIC
> /usr/local/builds/src/blas/libfblas.a: could not read symbols: Bad value
> collect2: ld returned 1 exit status
> /usr/lib64/gcc-lib/x86_64-suse-linux/3.3.5/../../../../x86_64-suse-linux/bin/ld:
> /usr/local/builds/src/blas/libfblas.a(dgemm.o): relocation R_X86_64_32
> against `a local symbol' can not be used when making a shared object;
> recompile with -fPIC
> /usr/local/builds/src/blas/libfblas.a: could not read symbols: Bad value
> collect2: ld returned 1 exit status
> error: Command "/usr/bin/g77 -shared
> build/temp.linux-x86_64-2.4/scipy/corelib/lapack_lite/lapack_litemodule.o
> -L/usr/local/builds/src/blas -Lbuild/temp.linux-x86_64-2.4 -lflapack_src
> -lfblas -lg2c -o build/lib.linux-x86_64-2.4/scipy/lib/lapack_lite.so"
> failed with exit status 1
> removed scipy/base/__svn_version__.py
> removed scipy/base/__svn_version__.pyc
> removed scipy/f2py/__svn_version__.py
> removed scipy/f2py/__svn_version__.pyc
>
> How can I fix these problems ?
>
> I strictly followed the instructions given at
> http://www.scipy.org/documentation/buildatlas4scipy.txt
>
> Are there special instructions how to build newcore/newscipy on 64 bit
> machines ?
This is a known problem on 64 Bit machines.
http://www.scipy.org/mailinglists/mailman?fn=scipy-user/2005-February/004066.html
http://www.scipy.net/pipermail/scipy-user/2005-September/005265.html
Below I post my complete notes on the installation
of **old** scipy on 64 Bit.
The only difference is that you want to use newscipy/newcore.
Everything else should be the same
(apart from fftw3, it seems that only fftw2 is supported in newscipy
at the moment.
Also no guarantee on performance for that one).
Note that the whole text is written in ReSt, so it
should be no problem to use the text or fragments
for the scipy wiki ...
HTH, Arnd
Get the sources
===============
Create directories::
mkdir -p INSTALL_PYTHON/Sources
cd INSTALL_PYTHON/Sources
`gcc <http://gcc.gnu.org/>`_::
wget ftp://ftp.gwdg.de/pub/misc/gcc/releases/gcc-3.4.4/gcc-3.4.4.tar.bz2
python and docu::
wget ftp://ftp.python.org/pub/python/2.4.2/Python-2.4.2.tar.bz2
wget http://www.python.org/ftp/python/doc/2.4.2/html-2.4.2.tar.bz2
Ipython via svn::
svn co http://ipython.scipy.org/svn/ipython/ipython/trunk ipython
zip -9r ipython ipython
rm -rf ipython
Numeric::
(press <enter> for CVS password)
cvs -d:pserver:anonymous at cvs.sourceforge.net:/cvsroot/numpy login
cvs -z3 -d:pserver:anonymous at cvs.sourceforge.net:/cvsroot/numpy co -P
Numerical
zip -9r Numerical Numerical
rm -rf Numerical
f2p2e::
(CVS password: guest)
cvs -d :pserver:anonymous at cens.ioc.ee:/home/cvs login
cvs -z6 -d :pserver:anonymous at cens.ioc.ee:/home/cvs checkout f2py2e
zip -9r f2py2e f2py2e
rm -rf f2py2e
`fftw <http://www.fftw.org/>`_::
wget http://www.fftw.org/fftw-3.0.1.tar.gz
`ATLAS <http://math-atlas.sourceforge.net/>_`::
wget
http://cesnet.dl.sourceforge.net/sourceforge/math-atlas/atlas3.6.0.tar.bz2
**OLD** scipy svn::
svn co http://svn.scipy.org/svn/scipy/trunk scipy
cd scipy
svn co http://svn.scipy.org/svn/scipy_core/trunk/ scipy_core
cd ..
zip -9r scipy scipy
rm -rf scipy
Directory ``Sources``
=====================
After this we have::
2032724 atlas3.7.11.tar.bz2
520527 f2py2e.zip
1946361 fftw-3.0.1.tar.gz
27565872 gcc-3.4.4.tar.bz2
1395537 html-2.4.2.tar.bz2
953884 ipython.zip
796402 Numerical.zip
7853169 Python-2.4.2.tar.bz2
16783889 scipy.zip
Environment variables
======================
::
export PHOME=/scr/python/
export INFODIR=${PHOME}/info:$INFODIR
export INFOPATH=${PHOME}/info:$INFOPATH
export MANPATH=${PHOME}/man:$MANPATH
export LD_LIBRARY_PATH=${PHOME}/lib64:${PHOME}/lib:$LD_LIBRARY_PATH
export LD_RUN_PATH=${PHOME}/lib64:${PHOME}/lib:$LD_RUN_PATH
export PYTHONDOCS=${PHOME}/docs/Python-Docs-2.4.2/
export PATH=${PHOME}/bin:${PATH}
export PAGER="less -R"
export CFLAGS=-fPIC
For the installation::
cd ..
mkdir CompileDir
cd CompileDir
(this should be on the same level as ``Sources``)
gcc
===
::
tar xjf ../Sources/gcc-3.4.4.tar.bz2
mkdir gcc-build
cd gcc-build
../gcc-3.4.4/configure --prefix=${PHOME} --enable-shared
--enable-threads=posix --enable-__cxa_atexit --enable-clocale=gnu
--enable-languages=c,c++,f77,objc
make
make install
cd ${PHOME}/bin/
ln -s gcc ${PHOME}/bin/cc
cd -
Python
======
::
tar xjf ../Sources/Python-2.4.2.tar.bz2
cd Python-2.4.2
./configure --prefix=${PHOME}
make
make install
make test
255 tests OK.
35 tests skipped:
test_aepack test_al test_applesingle test_bsddb185 test_bsddb3
test_cd test_cl test_codecmaps_cn test_codecmaps_hk
test_codecmaps_jp test_codecmaps_kr test_codecmaps_tw test_curses
test_dl test_gl test_imageop test_imgfile test_linuxaudiodev
test_macfs test_macostools test_nis test_normalization
test_ossaudiodev test_pep277 test_plistlib test_rgbimg
test_scriptpackages test_socket_ssl test_socketserver
test_sunaudiodev test_timeout test_urllib2net test_urllibnet
test_winreg test_winsound
Those skips are all expected on linux2.
cd ..
IPython
=======
::
unzip ../Sources/ipython.zip
cd ipython
python setup.py install
cd ..
rehash
# default Farben ipython ..., no question on CTRL-D::
xemacs -nw
/scr/python/lib/python2.4/site-packages/IPython/UserConfig/ipythonrc
#colors Linux
colors LightBG
confirm_exit 0
ATLAS and BLAS/LAPACK
=====================
See: http://www.scipy.org/documentation/buildatlas4scipy.txt
BLAS
----
::
export BLAS_SRC=${HOME}/INSTALL_PYTHON/CompileDir/BLAS
export LAPACK_SRC=${HOME}/INSTALL_PYTHON/CompileDir/LAPACK
mkdir -p $BLAS_SRC
cd $BLAS_SRC
wget http://www.netlib.org/blas/blas.tgz
tar xzf blas.tgz
g77 -fno-second-underscore -O2 -fPIC -c *.f
ar r libfblas.a *.o
ranlib libfblas.a
rm -rf *.o
LAPACK
------
::
mkdir -p $LAPACK_SRC
cd $LAPACK_SRC/..
wget http://www.netlib.org/lapack/lapack.tgz
tar xzf lapack.tgz
cd $LAPACK_SRC
cp INSTALL/make.inc.LINUX make.inc # on LINUX
Edit ``make.inc``::
OPTS = -funroll-all-loops -O3 -m64 -fno-second-underscore -fPIC
NOOPT = -m64 -fno-second-underscore -fPIC
and do::
make lapacklib
cd ..
ATLAS
-----
Just follow::
export BLAS=$BLAS_SRC/libfblas.a
tar xjf ../Sources/atlas3.7.11.tar.bz2
cd ATLAS
make xconfig
./xconfig -F f "-fomit-frame-pointer -O -fno-second-underscore -fPIC" -F
c " -fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC" -F m
"-fomit-frame-pointer -O -mfpmath=387 -m64 -fPIC" -b $BLAS
Settings:
- 64 Bit
- Posix threads
- don't stop
- use express setup? [y]:
- Enter f77 compiler [g77]:
Enter F77 Flags [-fomit-frame-pointer -O -m64]:
- Tune the Level 1 BLAS? [y]:
Gives for the compile commands::
F77 = /scr/python3//bin/g77 -fomit-frame-pointer -O
-fno-second-underscore -fPIC
CC = /scr/python3//bin/gcc -fomit-frame-pointer -O -mfpmath=387 -m64
-fPIC
MCC = /scr/python3//bin/gcc -fomit-frame-pointer -O -mfpmath=387 -m64
-fPIC
Have a couple of cups of coffee while doing::
make install arch=Linux_HAMMER64SSE3_2
make sanity_test arch=Linux_HAMMER64SSE3_2
make ptsanity_test arch=Linux_HAMMER64SSE3_2
Make optimized LAPACK library::
cd lib//Linux_HAMMER64SSE3_2/
mkdir tmp; cd tmp
ar x ../liblapack.a
cp ../liblapack.a ../liblapack.a_SAVE
cp ../../../../LAPACK/lapack_LINUX.a ../liblapack.a
ar r ../liblapack.a *.o
cd ..; rm -rf tmp
ls -lh liblapack.a
-rw-r--r-- 1 abaecker users 11M 2005-10-04 17:32 liblapack.a
cp *.a $PHOME/lib64
cd ../../
cp include/{cblas.h,clapack.h} $PHOME/include
cd ..
fftw
====
::
tar xzf ../Sources/fftw-3.0.1.tar.gz
cd fftw-3.0.1/
./configure CFLAGS=-fPIC --prefix=$PHOME
make
make install
Attempt with optimization::
./configure CFLAGS="-fPIC -O3 -fomit-frame-pointer -fno-schedule-insns
-fstrict-aliasing -mpreferred-stack-boundary=4" --prefix=$PHOME
Numerical
=========
::
unzip ../Sources/Numerical.zip
cd Numerical
emacs customize.py
if 1:
use_system_lapack = 1
lapack_library_dirs = ['/scr/python//lib64/']
lapack_libraries = ['lapack', 'cblas', 'f77blas', 'atlas', 'g2c']
if 1:
use_dotblas = 1
dotblas_include_dirs = ['/scr/python3/include/atlas']
dotblas_library_dirs = lapack_library_dirs
dotblas_libraries = ['cblas', 'atlas', 'g2c']
python setup.py build | tee ../Numeric_install_log.txt
python setup.py install | tee ../Numeric_install_log.txt
cd ..
scipy
=====
f2py
----
::
unzip ../Sources/f2py2e.zip
cd f2py2e
make install
cd ..
scipy itself
------------
::
unzip ../Sources/scipy.zip
cd scipy
export ATLAS=/scr/python/lib64:/scr/python/include
cd scipy_core/scipy_distutils
put the following into site.cfg::
[x11]
library_dirs = /usr/X11R6/lib64
include_dirs = /usr/X11R6/include
Then install first ``scipy_distutils``::
python setup.py install
cd ../..
Get ``scipy_systeminfo.txt`` for report on installed packages::
python scipy_core/scipy_distutils/system_info.py >
../scipy_systeminfo.txt
Install the beast::
python setup.py build | tee ../scipy_install_log.txt
python setup.py install
Get ``xplt`` working::
emacs /scr/python/lib/python2.4/site-packages/scipy/xplt/slice3.py
routine _construct3
change
mask = find_mask (below, _node_edges3 [itype])
to
mask = find_mask (below.astype("b"), _node_edges3 [itype].astype("b"))
Test::
cd
# Start python and run as tests:
# (level 10 will take a while ...)
import scipy
scipy.test(1,verbosity=10)
scipy.test(10,verbosity=10)
Only error::
======================================================================
FAIL: check_round (scipy.special.basic.test_basic.test_round)
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/scr/python/lib/python2.4/site-packages/scipy/special/tests/test_basic.py",
line 1789, in check_round
assert_array_equal(rnd,rndrl)
File "/scr/python/lib/python2.4/site-packages/scipy_test/testing.py",
line 715, in assert_array_equal
assert cond,\
AssertionError:
Arrays are not equal (mismatch 25.0%):
Array 1: [10 10 11 11]
Array 2: [10 10 10 11]
----------------------------------------------------------------------
Ran 1279 tests in 70.833s
More information about the SciPy-Dev
mailing list