[SciPy-User] fancy indexation of sparse matrices is terrible slow

[Dmitrey]

От кого: Nathan Bell <wnbell at gmail.com>  
  
  The CSR and CSC formats also support fast fancy indexing. Something like:  
>>> A = ... some sparse matrix  
>>> csr_matrix(A)[I,J]  
ought to work well.    
I have tried all 3, lil works faster than any others, but casting to numpy ndarray and invoking it on the dense array obtained works several times faster, that is extremely important for me. Thus on my problem with sparse matrix of shape 151* 1178  
each time casting it to dense before applying fancy indexing makes numerical optimization ipopt solver works faster from 130 to 60 sec. Here's the code that demonstrates the difference:  
  
from scipy.sparse import lil_matrix, find  
from numpy import where, zeros, prod, vstack, ones  
from time import time  
n, m = 1000, 20000  
M = vstack((zeros((n, m)), ones((25, m))))  
I, J = where(M)  
  
# 1: time for lil  
M = lil_matrix(M)  
print 'sparsity of the matrix:', float(M.size)/prod(M.shape)  
  
t = time()  
M[I, J]  
print('time elapsed with lil: %f' % (time()-t))  
  
# 2: time for dense  
t = time()  
M = M.A # even with this time elapsed is less  
M[I, J]  
print('time elapsed with ndarray: %f' % (time()-t))  
  
# output:  
# sparsity of the matrix: 0.0243902439024  
# time elapsed with lil: 17.006631  
# time elapsed with ndarray: 0.710046  
# as for csr and csc fancy indexation it works several times slower  
  
So, I have filed a ticket for it.  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20091214/f02b8a35/attachment.html>