[Numpy-discussion] Building numpy on Mac OS X 10.6, i386 (no ppc) 32/64bit: Error in Fortran tests due to ppc64

Mon Aug 16 06:17:13 EDT 2010

Hello!

At first, I'd like to say thanks to the numpy/scipy team and all contributors. Great software!

On Snow Leopard, aka Mac OS X 10.6.4 (server) I managed to build numpy 2.0.0.dev8636 (and scipy 0.9.0.dev6646) for arch i386 in combined 32/64bit against MacPorts python27 (No ppc here!).

All tests pass (yeha!), except for the fortran related ones. I think there is an issue with detecting the right arch. My numpy and python are both i386 32/64 bit but now ppc.

Only these tests fail, all others pass:
test_callback.TestF77Callback.test_all ... ERROR
test_mixed.TestMixed.test_all ... ERROR
test_return_character.TestF77ReturnCharacter.test_all ... ERROR
test_return_character.TestF90ReturnCharacter.test_all ... ERROR
test_return_complex.TestF77ReturnComplex.test_all ... ERROR
test_return_complex.TestF90ReturnComplex.test_all ... ERROR
test_return_integer.TestF77ReturnInteger.test_all ... ERROR
test_return_integer.TestF90ReturnInteger.test_all ... ERROR
test_return_logical.TestF77ReturnLogical.test_all ... ERROR
test_return_logical.TestF90ReturnLogical.test_all ... ERROR
test_return_real.TestCReturnReal.test_all ... ok
test_return_real.TestF77ReturnReal.test_all ... ERROR
test_return_real.TestF90ReturnReal.test_all ... ERROR
[...]
----------------------------------------------------------------------
Ran 2989 tests in 47.008s
FAILED (KNOWNFAIL=4, SKIP=1, errors=12)

Some more information (Perhaps I did some known mistake in those steps? Details at the end of this mail):
o      Mac OS X 10.6.4 (intel Core 2 duo)
o      Python 2.7 (r27:82500, Aug 15 2010, 12:19:40) 
         [GCC 4.2.1 (Apple Inc. build 5659) + GF 4.2.4] on darwin
o      gcc --version
         i686-apple-darwin10-gcc-4.2.1 (GCC) 4.2.1 (Apple Inc. build 5664)
o      gfortran --version
        GNU Fortran (GCC) 4.2.1 (Apple Inc. build 5659) + GF 4.2.4
        from gfortran from http://r.research.att.com/tools/
o      I used the BLAS/LAPACK that is provided by Apple's Accelerate framework.  
o      environment:
        export CFLAGS="-arch i386 -arch x86_64"
        export FFLAGS="-m32 -m64"
        export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch i386 -arch x86_64 -framework Accelerate"
o      bulid:
        python setup.py build --fcompiler=gnu95

I have not found a matching ticket in trac. Should I open one or did I something very stupid during the build process? Thanks!

Samuel

PS: I failed to succeed in the first shot with python.org's official fat precompiled .dmg-file release (ppc/i386 32/64 bit), so I used MacPorts. Later today, I'll try again to compile against python.org because I think numpy/scipy recommends that version.

For completeness, here are my build steps:

o   Building numpy/scipy from source:
    http://scipy.org/Installing_SciPy/Mac_OS_X:
    - Make sure XCode is installed with the Development target 10.4 SDK
    - Download and install gfortran from http://r.research.att.com/tools/
    - svn co http://svn.scipy.org/svn/numpy/trunk numpy
    - svn co http://svn.scipy.org/svn/scipy/trunk scipy
    - sudo port install fftw-3
    - sudo port install suitesparse
    - sudo port install swig-python
    - mkdir scipy_numpy; cd scipy_numpy
    - cd numpy
    - cp site.cfg.example site.cfg
    - You may want to copy the site.cfg to ~/.numpy-site.cfg
    - Edit site.cfg to contain only the following:
         [DEFAULT]
         library_dirs = /opt/local/lib
         include_dirs = /opt/local/include
         [amd]
         amd_libs = amd
         [umfpack]
         umfpack_libs = umfpack
         [fftw]
         libraries = fftw3
    - export MACOSX_DEPLOYMENT_TARGET=10.6
    - export CFLAGS="-arch i386 -arch x86_64"
    - export FFLAGS="-m32 -m64"
    - export LDFLAGS="-Wall -undefined dynamic_lookup -bundle -arch i386 -arch x86_64 -framework Accelerate"
    - export PYTHONPATH="/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/"
    - python setup.py build --fcompiler=gnu95
    - sudo python setup.py install
    - cd ..
    - cd scipy
    - sed  's|include <\(umfpack[^\.]*\.h\)>|include </opt/local/include/ufsparse/\1>|g' <scipy/sparse/linalg/dsolve/umfpack/umfpack.i  >scipy/sparse/linalg/dsolve/umfpack/___tmp.i
    - mv scipy/sparse/linalg/dsolve/umfpack/umfpack.i scipy/sparse/linalg/dsolve/umfpack/umfpack.old
    - mv scipy/sparse/linalg/dsolve/umfpack/___tmp.i scipy/sparse/linalg/dsolve/umfpack/umfpack.i
    - python setup.py build --fcompiler=gnu95
    - cd
    - python
      import numpy; numpy.test()
      import scipy; scipy.test()

A short excerpt of  numpy.test()'s output:

======================================================================
ERROR: test_return_real.TestF90ReturnReal.test_all
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose-0.11.4-py2.7.egg/nose/case.py", line 367, in setUp
    try_run(self.inst, ('setup', 'setUp'))
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/nose-0.11.4-py2.7.egg/nose/util.py", line 491, in try_run
    return func()
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/tests/util.py", line 341, in setUp
    module_name=self.module_name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/tests/util.py", line 73, in wrapper
    memo[key] = func(*a, **kw)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/tests/util.py", line 156, in build_code
    module_name=module_name)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/tests/util.py", line 73, in wrapper
    memo[key] = func(*a, **kw)
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/numpy/f2py/tests/util.py", line 128, in build_module
    % (cmd[4:], asstr(out)))
RuntimeError: Running f2py failed: ['-m', '_test_ext_module_5403', '/var/folders/sy/syxH3HBVEUyeFZOejvLLp+++-+2/-Tmp-/tmpJVSKpT/tmpHRbp1O.f90']
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building extension "_test_ext_module_5403" sources
f2py options: []
f2py:> /var/folders/sy/syxH3HBVEUyeFZOejvLLp+++-+2/-Tmp-/tmpOAl6fb/src.macosx-10.6-x86_64-2.7/_test_ext_module_5403module.c

[…snip…]

build_src: building npy-pkg config files
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
customize NAGFCompiler
Could not locate executable f95
customize AbsoftFCompiler
Could not locate executable f90
Could not locate executable f77
customize IBMFCompiler
Could not locate executable xlf90
Could not locate executable xlf
customize IntelFCompiler
Could not locate executable ifort
Could not locate executable ifc
customize GnuFCompiler
Could not locate executable g77
customize Gnu95FCompiler
Found executable /usr/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using build_ext
building '_test_ext_module_5403' extension
compiling C sources
C compiler: /usr/bin/gcc-4.2 -fno-strict-aliasing -fno-common -dynamic -pipe -O2 -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes

ortran f77 compiler: /usr/bin/gfortran  […snip…]
Fortran f90 compiler: /usr/bin/gfortran  […snip…]
Fortran fix compiler: /usr/bin/gfortran […snip…]

[…snip…]

ld: warning: in /var/folders/xxx/src.macosx-10.6-x86_64-2.7/_test_ext_module_5403module.o, file was built for unsupported file format which is not the architecture being linked (i386)

[…snip…]

ld: warning: in /usr/lib/libSystem.dylib, missing required architecture ppc64 in file
ld: in /usr/lib/libSystem.B.dylib, missing required architecture ppc64 in file for architecture ppc64
collect2: ld returned 1 exit status
lipo: can't open input file: /var/folders/sy/syxH3HBVEUyeFZOejvLLp+++-+2/-Tmp-//ccXwmeM6.out (No such file or directory)
error: Command "/usr/bin/gfortran -Wall -arch ppc -arch i686 -arch x86_64 -arch ppc64 -Wall -undefined dynamic_lookup -bundle /var/folders/xxx/src.macosx-10.6-x86_64-2.7/_test_ext_module_5403module.o /var/folders/xxx/src.macosx-10.6-x86_64-2.7/fortranobject.o /var/folders/sy/syxH3HBVEUyeFZOejvLLp+++-+2/-Tmp-/tmpOAl6fb/var/folders/sy/syxH3HBVEUyeFZOejvLLp+++-+2/-Tmp-/tmpJVSKpT/tmpHRbp1O.o /var/folders/xxx/src.macosx-10.6-x86_64-2.7/_test_ext_module_5403-f2pywrappers2.o -lgfortran -o ./_test_ext_module_5403.so" failed with exit status 1

[Note, i replace a long temp path by xxx just in this mail to make it more readable.]

----------------------------------------------------------------------
Ran 2989 tests in 47.008s

FAILED (KNOWNFAIL=4, SKIP=1, errors=12)

Btw: scipy.test():
Ran 5340 tests in 711.401s
FAILED (KNOWNFAIL=14, SKIP=31, errors=3, failures=3)