[SciPy-Dev] Why does forcing Numpy to compile with "sizeof(long double) == 8" fails tests?

[Gustavo Serra Scalet]

> -----Original Message-----
> From: SciPy-Dev [mailto:scipy-dev-bounces at scipy.org] On Behalf Of
> Sebastian Berg
> Sent: sexta-feira, 7 de outubro de 2016 04:04
> To: scipy-dev at scipy.org
> Subject: Re: [SciPy-Dev] Why does forcing Numpy to compile with
> "sizeof(long double) == 8" fails tests?
> 
> On Do, 2016-10-06 at 22:21 +0000, Gustavo Serra Scalet wrote:
> > Hi,
> >
> > I'm checking how Numpy performs on a POWER8 machine and I noticed that
> > some tests related to machar.py are failing:
> > =====================================================================
> > =
> > ERROR: test_singleton (test_getlimits.TestLongdouble)
> > -------------------------------------------------------------------
> > ---
> > Traceback (most recent call last):
> >   File "/home/gut/python/venv3.5-b/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+33d850d-py3.4-linux-
> > ppc64le.egg/numpy/core/tests/test_getlimits.py", line 41, in
> > test_singleton
> >     ftype = finfo(longdouble)
> >   File "/home/gut/python/venv3.5-b/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/getlimits.py",
> > line 124, in __new__
> >     obj = object.__new__(cls)._init(dtype)
> >   File "/home/gut/python/venv3.5-b/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/getlimits.py",
> > line 154, in _init
> >     'numpy %s precision floating point number' % precname)
> >   File "/home/gut/python/venv3.5-b/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py",
> > line 114, in __init__
> >     self._do_init(float_conv, int_conv, float_to_float, float_to_str,
> > title)
> >   File "/home/gut/python/venv3.5-b/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+33d850d-py3.4-linux-ppc64le.egg/numpy/core/machar.py",
> > line 127, in _do_init
> >     a = a + a
> > RuntimeWarning: overflow encountered in add
> >
> > FAILED (KNOWNFAIL=4, SKIP=10, errors=4)
> >
> >
> > That happened because native long double actually uses 2 doubles[1]
> > (that method is called IBM Extended double[2]).
> >
> > To avoid that, I changed the compile flags to use -mlong-double-64 [3]
> > and despite solving those errors, now I have some of these
> > failures:
> > =====================================================================
> > =
> > FAIL: test_in_from_2casttype
> > (test_array_from_pyobj.test_LONGDOUBLE_gen)
> > -------------------------------------------------------------------
> > ---
> > Traceback (most recent call last):
> >   File "/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+7a962d1-py3.4-linux-
> > ppc64le.egg/numpy/f2py/tests/test_array_from_pyobj.py", line 319, in
> > test_in_from_2casttype
> >     a = self.array([len(self.num2seq)], intent.in_, obj)
> >   File "<string>", line 6, in <lambda>
> >   File "/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+7a962d1-py3.4-linux-
> > ppc64le.egg/numpy/f2py/tests/test_array_from_pyobj.py", line 275, in
> > __init__
> >     assert_(self.has_shared_memory())
> >   File "/home/gut/python/venv3.5/lib/python3.4/site-packages/numpy-
> > 1.12.0.dev0+7a962d1-py3.4-linux-ppc64le.egg/numpy/testing/utils.py",
> > line 90, in assert_
> >     raise AssertionError(smsg)
> > AssertionError
> >
> > FAILED (KNOWNFAIL=6, SKIP=16, failures=3)
> >
> >
> > I wanted to know if there is a better approach to solve this issue. I
> > guess it supports 'long double == double', right?
> >
> 
> I would have a first look at the test, it is completely plausible that
> the tests are skipped on windows or so, but the check to skip it does
> not actually look at the long double size. So, I would not rule out the
> tests are simply invalid in your setting and are run because nobody
> tried this before.
> 
> - Sebastian

The tests were not being skipped by anybody but there was a condition to test longdoubles inside of the module that is not expecting it to be 8 bytes as double:
https://github.com/PPC64/numpy/commit/f24d6278ca774524828fbd79a11089f3789c0916

The main reason might be due to how the architecture is handling alignment of longdouble. I don't know. I hope to have some review of it on this PR:
https://github.com/numpy/numpy/pull/8134

Thanks for you feedback Sebastian.

> 
> 
> > Thanks in advance!
> >
> >
> > Environment:
> > Repository head at 33d850d522b3a1940c3ad06b63c79a466def11e9 (from
> > git://github.com/numpy/numpy.git) + patch on [3]
> > Python-3.5.2
> >
> >
> > [1] https://github.com/gcc-mirror/gcc/blob/3ff1ad36ed4ee3cc490e6178db
> > 87b1b8f2326c61/libgcc/config/rs6000/ibm-ldouble.c#L36-L43
> > [2] https://gcc.gnu.org/wiki/Ieee128PowerPC
> > [3] https://github.com/PPC64/numpy/commit/be15f1073802ab127162c5dcdd9
> > 95de7f6452c78
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at scipy.org
> > https://mail.scipy.org/mailman/listinfo/scipy-dev
> >