[SciPy-Dev] Schur decomposition test failure under Python 2.5
Bruce Southey
bsouthey at gmail.com
Tue Nov 8 10:55:36 EST 2011
On 11/07/2011 04:52 PM, Ralf Gommers wrote:
>
>
> On Mon, Nov 7, 2011 at 10:22 PM, Ralf Gommers
> <ralf.gommers at googlemail.com <mailto:ralf.gommers at googlemail.com>> wrote:
>
>
>
> On Mon, Nov 7, 2011 at 3:47 AM, Bruce Southey <bsouthey at gmail.com
> <mailto:bsouthey at gmail.com>> wrote:
>
> On Sun, Nov 6, 2011 at 2:22 AM, Ralf Gommers
> <ralf.gommers at googlemail.com
> <mailto:ralf.gommers at googlemail.com>> wrote:
> >
> >
> > On Sun, Nov 6, 2011 at 3:26 AM, Bruce Southey
> <bsouthey at gmail.com <mailto:bsouthey at gmail.com>> wrote:
> >>
> >> On Sat, Nov 5, 2011 at 1:40 PM, Ralf Gommers
> >> <ralf.gommers at googlemail.com
> <mailto:ralf.gommers at googlemail.com>> wrote:
> >> > Hi,
> >> >
> >> > There's a problem with schur(.., sort='lhp') under Python
> 2.5 that seems
> >> > to
> >> > be related to the Lapack function gees:
> >> > http://projects.scipy.org/scipy/ticket/1555
> >> >
> >> > It would be great if someone could have a look at this
> for the 0.10.0
> >> > release, but if not I'll mark it as a knownfailure
> because it's not a
> >> > regression.
> >> >
> >> > Ralf
> >> >
> >> >
> >> > _______________________________________________
> >> > SciPy-Dev mailing list
> >> > SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
> >> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >> >
> >> >
> >> I have only tested the RC on my 32-bit Windows install
> using the
> >> provided binary.
> >> I get the test_decomp.TestSchur and syntax error in
> test_distributions.py.
> >>
> >> But buried in the test output (when run within command line
> Python but
> >> not under IDLE) is this:
> >> "capi_return is NULL
> >> Call-back cb_dselect_in_dgees__user__routines failed."
> >>
> > That's the cause of the TestSchur failure.
> >
> >>
> >> This also appears with the current dev version under Linux.
> >>
> >> Bruce
> >>
> >> $ python2.5 test_decomp.py
> >>
> >>
> ................................................................................................................................................capi_return
> >> is NULL
> >> Call-back cb_dselect_in_dgees__user__routines failed.
> >>
> >>
> E.....................................................................................................................................E
> >>
> ======================================================================
> >> ERROR: test_sort (test_decomp.TestSchur)
> >>
> ----------------------------------------------------------------------
> >> Traceback (most recent call last):
> >> File
> >>
> "/home/bsouthey/python/scipystuff/git/scipy/scipy/linalg/tests/test_decomp.py",
> >> line 1498, in test_sort
> >> s,u,sdim = schur(a,sort='lhp')
> >> File
> >>
> "/usr/local/lib/python2.5/site-packages/scipy/linalg/decomp_schur.py",
> >> line 118, in schur
> >> sort_t=sort_t)
> >> File
> >>
> "/usr/local/lib/python2.5/site-packages/scipy/linalg/decomp_schur.py",
> >> line 106, in <lambda>
> >> sfunction = lambda x: (x.real < 0.0)
> >> AttributeError: 'float' object has no attribute 'real'
> >>
> >>
> ======================================================================
> >> ERROR: test_decomp.test_lapack_misaligned
> >>
> ----------------------------------------------------------------------
> >> Traceback (most recent call last):
> >> File
> "/usr/local/lib/python2.5/site-packages/nose/case.py", line
> >> 186, in runTest
> >> self.test(*self.arg)
> >> File
> >>
> "/usr/local/lib/python2.5/site-packages/numpy/testing/decorators.py",
> >> line 213, in knownfailer
> >> raise KnownFailureTest, msg
> >> KnownFailureTest: Ticket #1152, triggers a segfault in rare
> cases.
> >>
> >>
> ----------------------------------------------------------------------
> >
> > This one is new. Is this python 2.5 on Linux? Is the error
> reproduceable?
> >
> > Ralf
> >
> Sorry as it is a known failure just that running the test file
> does
> not pickup the declaration.
>
>
> Odd. @dec.knownfailureif(True, "...") doesn't work. Code looks
> fine to me, can't reproduce it.
>
> I am presuming others will test Python2.6, Python2.7 and
> Python3.1.
>
> This Python3.2 error:
> =====================================================================
> ERROR: Failure: AttributeError ('module' object has no
> attribute 'FileType')
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/lib/python3.2/site-packages/nose-1.0.0-py3.2.egg/nose/failure.py",
> line 37, in runTest
> raise self.exc_class(self.exc_val).with_traceback(self.tb)
> File
> "/usr/lib/python3.2/site-packages/nose-1.0.0-py3.2.egg/nose/loader.py",
> line 390, in loadTestsFromName
> addr.filename, addr.module)
> File
> "/usr/lib/python3.2/site-packages/nose-1.0.0-py3.2.egg/nose/importer.py",
> line 39, in importFromPath
> return self.importFromDir(dir_path, fqname)
> File
> "/usr/lib/python3.2/site-packages/nose-1.0.0-py3.2.egg/nose/importer.py",
> line 86, in importFromDir
> mod = load_module(part_fqname, fh, filename, desc)
> File
> "/usr/lib64/python3.2/site-packages/scipy/weave/__init__.py",
> line 22, in <module>
> from .blitz_tools import blitz
> File
> "/usr/lib64/python3.2/site-packages/scipy/weave/blitz_tools.py",
> line 6, in <module>
> from . import converters
> File
> "/usr/lib64/python3.2/site-packages/scipy/weave/converters.py",
> line 19, in <module>
> c_spec.file_converter(),
> File "/usr/lib64/python3.2/site-packages/scipy/weave/c_spec.py",
> line 74, in __init__
> self.init_info()
> File "/usr/lib64/python3.2/site-packages/scipy/weave/c_spec.py",
> line 264, in init_info
> self.matching_types = [types.FileType]
> AttributeError: 'module' object has no attribute 'FileType'
>
> ----------------------------------------------------------------------
>
> Due to weave not being py3k compatible. Perhaps we should raise a
> clearer error here.
>
>
> Is Python2.4 still being supported as there are 8 errors (see
> below)?
>
> It is. Just kind of hard to support it in practice with no one
> using it and no buildbot. Thanks for finding these errors.
>
>
> Bruce
>
> $ python2.4 -c "import scipy; scipy.test()"
> Running unit tests for scipy
> NumPy version 2.0.0.dev-93236a2
> NumPy is installed in /usr/local/lib/python2.4/site-packages/numpy
> SciPy version 0.10.0rc1
> SciPy is installed in /usr/local/lib/python2.4/site-packages/scipy
> Python version 2.4.6 (#1, Sep 13 2010, 15:54:12) [GCC 4.4.4
> 20100630
> (Red Hat 4.4.4-10)]
> nose version 0.11.2
> /usr/local/lib/python2.4/site-packages/scipy/maxentropy/__init__.py:19:
> DeprecationWarning:
> The scipy.maxentropy module is deprecated in scipy 0.10, and
> scheduled to be
> removed in 0.11.
>
> If you are using some of the functionality in this module and
> are of the
> opinion that it should be kept or moved somewhere - or you are
> even interested
> to maintain/improve this whole module - please ask on the
> scipy-dev mailing
> list.
>
> The logsumexp function has already been moved to scipy.misc.
> DeprecationWarning)
> ............................................................................................................................................................................................................................K............................................................................................................/usr/local/lib/python2.4/site-packages/scipy/interpolate/fitpack2.py:674:
> UserWarning:
> The coefficients of the spline returned have been computed as the
> minimal norm least-squares solution of a (numerically) rank
> deficient
> system (deficiency=7). If deficiency is large, the results may be
> inaccurate. Deficiency may strongly depend on the value of eps.
> warnings.warn(message)
> ....../usr/local/lib/python2.4/site-packages/scipy/interpolate/fitpack2.py:605:
> UserWarning:
> The required storage space exceeds the available storage
> space: nxest
> or nyest too small, or s too small.
> The weighted least-squares spline corresponds to the current
> set of
> knots.
> warnings.warn(message)
> ........................K..K................................................................................................................................................................................................................................................................................................................................................................................................................................................../usr/local/lib/python2.4/site-packages/scipy/io/wavfile.py:31:
> WavFileWarning: Unfamiliar format bytes
> warnings.warn("Unfamiliar format bytes", WavFileWarning)
> /usr/local/lib/python2.4/site-packages/scipy/io/wavfile.py:121:
> WavFileWarning: chunk not understood
> warnings.warn("chunk not understood", WavFileWarning)
> ...............................................................................................................................................................................................................................SSSSSS......SSSSSS......SSSS......................................................................................................................................................................................................capi_return
> is NULL
> Call-back cb_dselect_in_dgees__user__routines failed.
> E.....................................................................................................................................K......................................................................................................................................................................................................SSSSS............S............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................E...........................................K...............................................SSSSSSSSSSS.......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................K...............................................................K...........................................................................................E....................E.......E.....E.......E....................KK.............................................................................................................................................................................................................................................................................................................................................................................................................................................K.K.............................................................................................................................................................................................................................................................................................................................................................................................K........K..............SSSSSSS....................................................................................................................................................../usr/local/lib/python2.4/site-packages/scipy/stats/distributions.py:1258:
> DeprecationWarning: putmask has been deprecated. Use copyto with
> 'where' as the mask instead
> putmask(output,(1-cond0)*array(cond1,bool),self.badvalue)
> ....SE.....................................................................................................................................................................................................................................................................................................................................................................................................................
> ======================================================================
> ERROR: test_sort (test_decomp.TestSchur)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.4/site-packages/scipy/linalg/tests/test_decomp.py",
> line 1230, in test_sort
> s,u,sdim = schur(a,sort='lhp')
> File
> "/usr/local/lib/python2.4/site-packages/scipy/linalg/decomp_schur.py",
> line 118, in schur
> sort_t=sort_t)
> File
> "/usr/local/lib/python2.4/site-packages/scipy/linalg/decomp_schur.py",
> line 106, in <lambda>
> sfunction = lambda x: (x.real < 0.0)
> AttributeError: 'float' object has no attribute 'real'
>
>
> http://projects.scipy.org/scipy/ticket/1555
>
>
> ======================================================================
> ERROR: test_signaltools.TestHilbert2.test_bad_args
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/usr/local/lib/python2.4/site-packages/nose/case.py", line
> 186, in runTest
> self.test(*self.arg)
> File
> "/usr/local/lib/python2.4/site-packages/scipy/signal/tests/test_signaltools.py",
> line 708, in test_bad_args
> assert_raises(ValueError, hilbert2, x, N=(2,0))
> File
> "/usr/local/lib/python2.4/site-packages/numpy/testing/utils.py",
> line 1053, in assert_raises
> return nose.tools.assert_raises(*args,**kwargs)
> File "/usr/local/lib/python2.4/unittest.py", line 320, in
> failUnlessRaises
> callableObj(*args, **kwargs)
> File
> "/usr/local/lib/python2.4/site-packages/scipy/signal/signaltools.py",
> line 746, in hilbert2
> elif len(N) != 2 or any(n <= 0 for n in N):
> NameError: name 'any' is not defined
>
>
> Will push a fix for that in a minute.
>
>
> ======================================================================
> ERROR: adding a dense matrix to a sparse matrix
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/tests/test_base.py",
> line 519, in test_add_dense
> sum1 = self.dat + self.datsp
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/dok.py",
> line 133, in __getitem__
> raise TypeError('index must be a pair of integers or slices')
> TypeError: index must be a pair of integers or slices
>
> ======================================================================
> ERROR: test_matmat_sparse (test_base.TestDOK)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/tests/test_base.py",
> line 417, in test_matmat_sparse
> assert_array_almost_equal( a2*bsp, a*b)
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/dok.py",
> line 133, in __getitem__
> raise TypeError('index must be a pair of integers or slices')
> TypeError: index must be a pair of integers or slices
>
> ======================================================================
> ERROR: test_radd (test_base.TestDOK)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/tests/test_base.py",
> line 279, in test_radd
> c = a + b
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/dok.py",
> line 133, in __getitem__
> raise TypeError('index must be a pair of integers or slices')
> TypeError: index must be a pair of integers or slices
>
> ======================================================================
> ERROR: test_rsub (test_base.TestDOK)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/tests/test_base.py",
> line 290, in test_rsub
> assert_array_equal((self.dat -
> self.datsp),[[0,0,0,0],[0,0,0,0],[0,0,0,0]])
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/dok.py",
> line 133, in __getitem__
> raise TypeError('index must be a pair of integers or slices')
> TypeError: index must be a pair of integers or slices
>
> ======================================================================
> ERROR: subtracting a dense matrix to/from a sparse matrix
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/tests/test_base.py",
> line 527, in test_sub_dense
> sum1 = 3*self.dat - self.datsp
> File
> "/usr/local/lib/python2.4/site-packages/scipy/sparse/dok.py",
> line 133, in __getitem__
> raise TypeError('index must be a pair of integers or slices')
> TypeError: index must be a pair of integers or slices
>
> These are odd. A dok_matrix can't be indexed with a single
> integer, which causes errors when we do ``obj + dokmatrix`` when
> obj is an ndarray or matrix. It looks to me like indexing with
> single integers should trigger a conversion to a dense array or
> something like that.
>
>
> dok_matrix.__radd__ (which has the todense() call) is called for
> Python >= 2.5, but not for 2.4. I can't find anything related to this
> in the "what's new in python 2.5" doc.
>
> Ralf
>
>
> Also I don't understand why it doesn't fail for other Python
> versions. Are numpy arrays somehow aware of sparse matrix details?
>
> If anyone with more knowledge of how this sparse matrix arithmetic
> is supposed to work can explain or have a look, that would be great.
>
> Ralf
>
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
Actually this Python 2.4 error has been here since 0.7.0 when the dok
format was introduced. But means there are no or very few Python2.4
scipy users or they have some way around this (calling todense() ) and
other issues.
I am totally ignorant here and probably why I gave up assuming I looked
at this before (as I usually try to find the cause of the failure).
Python 2.4 appears to call __getitem__ and but not __radd__ whereas
Python2.5 does not call __getitem__ but calls __radd__. So all I can
understand is that I can reproduce the error under 0.7.0 as follows.
Perhaps a bug should be filed and perhaps set to won't fix if Python2.4
support will be dropped 'soon' because this apparently has not been
previously been reported.
Bruce
$ python2.4
Python 2.4.6 (#1, Sep 13 2010, 15:54:12)
[GCC 4.4.4 20100630 (Red Hat 4.4.4-10)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> import scipy as sp
>>> import scipy.sparse as sparse
>>> print 'Numpy version=',np.__version__
Numpy version= 2.0.0.dev-93236a2
>>> print 'Scipy version=',sp.__version__
Scipy version= 0.7.0
>>> dmat= np.matrix([[1,0,0,2],[3,0,1,0],[0,2,0,0]],'d')
>>> print 'dense matrix\n', dmat
dense matrix
[[ 1. 0. 0. 2.]
[ 3. 0. 1. 0.]
[ 0. 2. 0. 0.]]
>>> smat=sparse.dok_matrix(dmat)
>>> print 'Sparse dok matrix\n', smat
Sparse dok matrix
(1, 2) 1.0
(0, 3) 2.0
(0, 0) 1.0
(1, 0) 3.0
(2, 1) 2.0
>>> sumd=dmat+smat.todense() # work around
>>> print 'use todense\n', sumd
use todense
[[ 2. 0. 0. 4.]
[ 6. 0. 2. 0.]
[ 0. 4. 0. 0.]]
>>> sum1=dmat+smat
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/local/lib/python2.4/site-packages/scipy/sparse/dok.py",
line 119, in __getitem__
raise TypeError('index must be a pair of integers or slices')
TypeError: index must be a pair of integers or slices
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20111108/05d8cba7/attachment.html>
More information about the SciPy-Dev
mailing list