[SciPy-Dev] Sporadic failures of tests of signal.correlate with dtype complex64

Mon Nov 29 18:52:52 EST 2010

On Tue, Nov 2, 2010 at 9:18 AM, Ralf Gommers <ralf.gommers at googlemail.com>wrote:

> On Tue, Nov 2, 2010 at 1:24 PM, Warren Weckesser
> <warren.weckesser at enthought.com> wrote:
> > On Mac OSX 10.5.8, I'm seeing occasional failures like the following:
> >
> > $ python -c "import scipy.signal; scipy.signal.test()"
> > Running unit tests for scipy.signal
> > NumPy version 1.5.0.dev8716
> > NumPy is installed in
> > /Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy
> > SciPy version 0.9.0.dev6856
> > SciPy is installed in
> > /Users/warren/tmp_py_install/lib/python2.6/site-packages/scipy
> > Python version 2.6.5 |EPD 6.2-2 (32-bit)| (r265:79063, May 28 2010,
> > 15:13:03) [GCC 4.0.1 (Apple Inc. build 5488)]
> > nose version 0.11.3
> >
> ................./Users/warren/tmp_py_install/lib/python2.6/site-packages/scipy/signal/filter_design.py:256:
> > BadCoefficients: Badly conditioned filter coefficients (numerator): the
> > results may be meaningless
> >   "results may be meaningless", BadCoefficients)
> >
> ......................................................F...............................................................................................................................................................................................................................................
> > ======================================================================
> > FAIL: test_rank1_same (test_signaltools.TestCorrelateComplex64)
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/scipy/signal/tests/test_signaltools.py",
> > line 606, in test_rank1_same
> >     assert_array_almost_equal(y, y_r)
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy/testing/utils.py",
> > line 774, in assert_array_almost_equal
> >     header='Arrays are not almost equal')
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy/testing/utils.py",
> > line 618, in assert_array_compare
> >     raise AssertionError(msg)
> > AssertionError:
> > Arrays are not almost equal
> >
> > (mismatch 10.0%)
> >  x: array([-6.76370811-8.55324841j,  0.68672836-4.2681613j ,
> >        -3.22760987-8.69287109j,  0.75051951-5.50820398j,
> >        -7.33016682-1.14685655j, -5.99573374+7.84123898j,...
> >  y: array([-6.76370859-8.55324745j,  0.68672895-4.2681613j ,
> >        -3.22761011-8.69286919j,  0.75051963-5.50820446j,
> >        -7.33016682-1.14685678j, -5.99573517+7.84123898j,...
> >
> > ----------------------------------------------------------------------
> > Ran 311 tests in 2.307s
> >
> > FAILED (failures=1)
> >
> > $ python -c "import scipy.signal; scipy.signal.test()"
> > Running unit tests for scipy.signal
> > NumPy version 1.5.0.dev8716
> > NumPy is installed in
> > /Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy
> > SciPy version 0.9.0.dev6856
> > SciPy is installed in
> > /Users/warren/tmp_py_install/lib/python2.6/site-packages/scipy
> > Python version 2.6.5 |EPD 6.2-2 (32-bit)| (r265:79063, May 28 2010,
> > 15:13:03) [GCC 4.0.1 (Apple Inc. build 5488)]
> > nose version 0.11.3
> >
> ................./Users/warren/tmp_py_install/lib/python2.6/site-packages/scipy/signal/filter_design.py:256:
> > BadCoefficients: Badly conditioned filter coefficients (numerator): the
> > results may be meaningless
> >   "results may be meaningless", BadCoefficients)
> >
> .......................................................F..............................................................................................................................................................................................................................................
> > ======================================================================
> > FAIL: test_rank1_same_old (test_signaltools.TestCorrelateComplex64)
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy/testing/decorators.py",
> > line 257, in _deprecated_imp
> >     f(*args, **kwargs)
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/scipy/signal/tests/test_signaltools.py",
> > line 641, in test_rank1_same_old
> >     assert_array_almost_equal(y, y_r)
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy/testing/utils.py",
> > line 774, in assert_array_almost_equal
> >     header='Arrays are not almost equal')
> >   File
> >
> "/Users/warren/tmp_py_install/lib/python2.6/site-packages/numpy/testing/utils.py",
> > line 618, in assert_array_compare
> >     raise AssertionError(msg)
> > AssertionError:
> > Arrays are not almost equal
> >
> > (mismatch 10.0%)
> >  x: array([ 2.46665049+2.02072477j, -7.42591763-0.54789257j,
> >         3.41454220-0.15863085j, -0.14030695+5.01129198j,
> >        -2.11230707+2.68583822j,  7.78784609+7.19434834j,...
> >  y: array([ 2.46665049+2.02072501j, -7.42591763-0.54789257j,
> >         3.41454196-0.15863061j, -0.14030659+5.01129246j,
> >        -2.11230707+2.68583822j,  7.78784752+7.19434786j,...
> >
> > ----------------------------------------------------------------------
> > Ran 311 tests in 2.623s
> >
> > FAILED (failures=1)
> >
> >
> > The above tests are part of a suite of tests that use random data, and
> > usually the tests all pass.  It took several tries to get the above
> > failures.
> >
> > I suspect the problem is simply that the default tolerance of
> > 'assert_array_almost_equal' is too small for the complex64 data type for
> > these tests.
> >
> > Could someone verify that they can reproduce those failures?
>
> Can't reproduce these.
>
> > Does simply increasing the tolerance of the test look like a reasonable
> fix?
>
> The default tolerance is decimal=6, which is not all that strict. I
> notice that only the longdouble tests fail while the single/double
> tests do not. This is very likely platform dependent, otherwise it
> would have been noticed before.
>

Actually, it was not the longdouble case that was failing; the test class
TestCorrelateComplex64 is the complex single precision case.

>
> Like Josef/Pauli say fixing the seed is one thing (preferably with a
> value that's failing for you). But then I would split the tests and
> only increase the tolerance of the longdouble version, perhaps even
> only on certain platforms.
>

In r6982, I modified the complex test cases of the correlate function to use
a decimal precision that depends on the data type; single, double and
longdouble are tested with 'decimal' being 5, 10 and 15, respectively.  This
means that the double and longdouble cases are now being tested with a
stricter tolerance than before.

I also added a line to set the seed to a value for which the tests had been
failing for me.

Warren
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20101129/3b0bbe59/attachment.html>