[Numpy-discussion] Revisiting numpy/scipy on 64 bit OSX
Chris Kees
christopher.e.kees at usace.army.mil
Fri Aug 22 10:00:19 EDT 2008
I've been experimenting with both a non-framework, non-universal 64-bit
build and a 4-way universal build of the python (2.6) trunk with numpy
1.1.1. The non-framework 64 build appears to give me exactly the same
results from numpy.test() as the standard 32-bit version (as well as
allowing large arrays like numpy.zeros((1000,1000,1000),'d') ), which is
<unittest._TextTestResult run=1300 errors=0 failures=0>
Our numerical models also seem to run fine with it using 8-16G. The 4-way
universal python gives the same results in 32-bit but when running in
64-bit I get an error in the tests below, which I haven't had time to look
at. It also gives the error
>>> a = numpy.zeros((1000,1000,1000),'d')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: dimensions too large.
Anyway, this all goes to say I'm interested in making sure numpy is
dependable in 64-bit on OSX 10.5 and preferably for 4-way universal builds,
but I haven't seen test failures like yours, at least not recently. In my
experience so far on the Mac it's still important to make sure Xcode and the
OS are up to date (or to use a completely segregated set of consistently
build gnu compilers like you did) and to make sure the fink and mac ports
stuff is not in your path before building python and numpy. Otherwise
configure and distutils end up making bad choices that I don't have the
skill to root out. I've had various versions of non-framework 64-bit builds
of python running with numpy and Numeric for a while now (a year maybe). The
framework and universal builds of python have only worked recently, and I'm
still not confident in them.
I don't know what's involved in the buildbots. If it's simple, doesn't take
much time to set up, and works behind a tight firewall I might be able to
set some up.
Chris
Numpy is installed in
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-package
s/numpy
Numpy version 1.1.1
Python version 2.6b2+ (trunk:65908, Aug 20 2008, 08:47:16) [GCC 4.0.1 (Apple
Inc. build 5484)]
Found 18/18 tests for numpy.core.tests.test_defmatrix
Found 3/3 tests for numpy.core.tests.test_errstate
Found 3/3 tests for numpy.core.tests.test_memmap
Found 290/290 tests for numpy.core.tests.test_multiarray
Found 72/72 tests for numpy.core.tests.test_numeric
Found 36/36 tests for numpy.core.tests.test_numerictypes
Found 12/12 tests for numpy.core.tests.test_records
Found 145/145 tests for numpy.core.tests.test_regression
Found 7/7 tests for numpy.core.tests.test_scalarmath
Found 2/2 tests for numpy.core.tests.test_ufunc
Found 16/16 tests for numpy.core.tests.test_umath
Found 63/63 tests for numpy.core.tests.test_unicode
Found 4/4 tests for numpy.distutils.tests.test_fcompiler_gnu
Found 5/5 tests for numpy.distutils.tests.test_misc_util
Found 2/2 tests for numpy.fft.tests.test_fftpack
Found 3/3 tests for numpy.fft.tests.test_helper
Found 25/25 tests for numpy.lib.tests.test__datasource
Found 10/10 tests for numpy.lib.tests.test_arraysetops
Found 1/1 tests for numpy.lib.tests.test_financial
Found 55/55 tests for numpy.lib.tests.test_function_base
Found 5/5 tests for numpy.lib.tests.test_getlimits
Found 6/6 tests for numpy.lib.tests.test_index_tricks
/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-package
s/numpy/lib/tests/test_io.py:11: SyntaxWarning: assertion is always true,
perhaps remove parentheses?
assert(c.readlines(),
Found 15/15 tests for numpy.lib.tests.test_io
Found 1/1 tests for numpy.lib.tests.test_machar
Found 4/4 tests for numpy.lib.tests.test_polynomial
Found 1/1 tests for numpy.lib.tests.test_regression
Found 49/49 tests for numpy.lib.tests.test_shape_base
Found 15/15 tests for numpy.lib.tests.test_twodim_base
Found 43/43 tests for numpy.lib.tests.test_type_check
Found 1/1 tests for numpy.lib.tests.test_ufunclike
Found 89/89 tests for numpy.linalg.tests.test_linalg
Found 3/3 tests for numpy.linalg.tests.test_regression
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/__init__.py", line 126, in test
return NumpyTest().test(*args, **kw)
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/testing/numpytest.py", line 579, in test
level, verbosity)
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/testing/numpytest.py", line 524, in _test_suite_from_all_tests
verbosity=verbosity)
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/testing/numpytest.py", line 417, in _get_suite_list
suite = obj(mthname)
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/ma/tests/test_core.py", line 36, in __init__
self.setUp()
File
"/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packag
es/numpy/ma/tests/test_core.py", line 49, in setUp
xf = numpy.where(m1, 1.e+20, x)
ValueError: invalid entry in choice array
On 8/21/08 4:15 PM, "Michael Abshoff" <michael.abshoff at googlemail.com>
wrote:
> Hi,
>
> I have been playing with 64 bit numpy/scipy on OSX 10.5 Intel again. I
> thought everything worked after the last time we discussed this, but I
> noticed that I had Scipy import failures in for example optimize since
> the gfortran I used creates 32 bit code per default.
>
> So I build a gcc 4.2.4 on OSX, then build python in 64 bit mode (fixing
> configure.in since Python assumes flags only available with the Apple
> gcc on Darwin), added a wrapper around gfortran that injects a "-m64"
> compile flag. Then I checkout out and build numpy (r5671), scipy (r4661)
> and the current nose release and ran the tests. I had some more failures
> than I expected (details below).
>
> Any thoughts on the failures? I am about to build ATLAS and a couple
> other dependencies on that box.
>
> William Stein is at Scipy 08 and there seems to be at least some
> interest in 64 bit OSX binaries. If anyone wants one let me know and I
> can put one together once Scipy 0.7 and Numpy 1.1.2 are out.
>
> Cheers,
>
> Michael
>
> numpy failed tests:
>
> ****************************************************************
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 29, in test_polynomial
> Failed example:
> p / q
> Expected:
> (poly1d([ 0.33333333]), poly1d([ 1.33333333, 2.66666667]))
> Got:
> (poly1d([ 0.333]), poly1d([ 1.333, 2.667]))
> **********************************************************************
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 51, in test_polynomial
> Failed example:
> p.integ()
> Expected:
> poly1d([ 0.33333333, 1. , 3. , 0. ])
> Got:
> poly1d([ 0.333, 1. , 3. , 0. ])
> **********************************************************************File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 53, in test_polynomialFailed example:
> p.integ(1)
> Expected:
> poly1d([ 0.33333333, 1. , 3. , 0. ])
> Got:
> poly1d([ 0.333, 1. , 3. , 0. ])
> **********************************************************************
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_polynomial.py",
> line 55, in test_polynomial
> Failed example:
> p.integ(5)
> Expected:
> poly1d([ 0.00039683, 0.00277778, 0.025 , 0. , 0.
> ,
> 0. , 0. , 0. ])
> Got:
> poly1d([ 0. , 0.003, 0.025, 0. , 0. , 0. , 0. , 0.
> ])
>
> ********************************
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_ufunclike.py",
> line 48, in test_ufunclike
> Failed example:
> U.log2(a)
> Expected:
> array([ 2.169925 , 1.20163386, 2.70043972])
> Got:
> array([ 2.17 , 1.202, 2.7 ])
> **********************************************************************
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_ufunclike.py",
> line 53, in test_ufunclike
> Failed example:
> U.log2(a, y)
> Expected:
> array([ 2.169925 , 1.20163386, 2.70043972])
> Got:
> array([ 2.17 , 1.202, 2.7 ])
> **********************************************************************
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/lib/tests
> /test_ufunclike.py",
> line 55, in test_ufunclike
> Failed example:
> y
> Expected:
> array([ 2.169925 , 1.20163386, 2.70043972])
> Got:
> array([ 2.17 , 1.202, 2.7 ])
> ======================================================================
> FAIL: test_umath.TestComplexFunctions.test_it
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/nose/case.py",
> line 182, in runTest
> self.test(*self.arg)
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/core/test
> s/test_umath.py",
> line 196, in test_it
> assert_almost_equal(fz.real, fr, err_msg='real part %s'%f)
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/testing/u
> tils.py",
> line 207, in assert_almost_equal
> assert round(abs(desired - actual),decimal) == 0, msg
> AssertionError:
> Items are not equal: real part <ufunc 'log10'>
> ACTUAL: -0.3010299956639812
> DESIRED: 0.0
>
>
> scipy:
>
> ======================================================================
> ERROR: test_nonsymmetric_modes (test_arpack.TestEigenNonSymmetric)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 204, in test_nonsymmetric_modes
> self.eval_evec(m,typ,k,which)
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 186, in eval_evec
> eval,evec=eigen(a,k,which=which,**kwds)
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/arpack.py",
> line 220, in eigen
> raise RuntimeError("Error info=%d in arpack"%info)
> RuntimeError: Error info=-8 in arpack
>
> ======================================================================
> ERROR: test_starting_vector (test_arpack.TestEigenNonSymmetric)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 214, in test_starting_vector
> self.eval_evec(self.symmetric[0],typ,k,which='LM',v0=v0)
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/tests/test_arpack.py",
> line 186, in eval_evec
> eval,evec=eigen(a,k,which=which,**kwds)
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/sparse/li
> nalg/eigen/arpack/arpack.py",
> line 220, in eigen
> raise RuntimeError("Error info=%d in arpack"%info)
> RuntimeError: Error info=-8 in arpack
> ======================================================================
> FAIL: test_implicit (test_odr.TestODR)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/scipy/odr/tests
> /test_odr.py",
> line 118, in test_implicit
> 2.0778813389755596e-03]]),
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/testing/u
> tils.py",
> line 304, in assert_array_almost_equal
> header='Arrays are not almost equal')
> File
> "/Users/mabshoff/64bit_numpy_scipy/lib/python2.5/site-packages/numpy/testing/u
> tils.py",
> line 289, in assert_array_compare
> assert cond, msg
> AssertionError:
> Arrays are not almost equal
>
> (mismatch 4.0%)
> x: array([[ 2.10892628e+00, -1.94376722e+00, 7.02635228e-02,
> -4.71752493e-02, 5.25155371e-02],
> [ -1.94376722e+00, 2.04814914e+00, -6.16004809e-02,...
> y: array([[ 2.10892746e+00, -1.94376864e+00, 7.02635509e-02,
> -4.71752674e-02, 5.25155759e-02],
> [ -1.94376864e+00, 2.04815092e+00, -6.16005159e-02,...
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion
More information about the NumPy-Discussion
mailing list