[SciPy-Dev] Enhancement Request: SuperLU_dist/SuperLU_mt replace current scipy.sparse.superLU

[Neil Godber]

Hi dev-list,

I've logged an enhancement request on github to bring the parallel versions
of SuperLU into scipy. As requested I'm circulating the enhancement here to
solicit comment/feedback/engagement.

Content of request is as follows (with light editing and links):

"Is your feature request related to a problem? Please describe.

The direct solution of sparse matrices is a common problem that arises
across many domains. Current scipy.sparse.linalg provides spsolve and splu
to solve such classes of problems. Neither solution appears to utilise more
than one or two threads leaving a lot of the available compute on modern
machines unused. Other solutions (MUMPS, Pardiso) exploit multicore
processors but are either minimally supported and/or locked behind
proprietary libraries.

Describe the solution you'd like
Fortunately SuperLU has two multithreaded implementations superLU_mt and
superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov)
<https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would
replace the current sequential implementation with one of the parallel
implementations to yield large performance improvements.

Describe alternatives you've considered
pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines
(github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues
associated with that for non Intel processors, unmaintained relatively to
scipy, PyTrilinos (implements all three superLU types but only available on
osx and linux PyTrilinos | Trilinos
<https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc /
petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three
superLU types but only available on osx and linux). All are relatively
inaccessible compared to ubiquity and accessibility of scipy.

Additional context (e.g. screenshots)
Anecdotally large number of projects use various non scipy sparse solvers
to achieve high level of performance. This comes with considerable time and
maintenance investment. It would be fantastic for scipy to provide a
'performance competitive' sparse direct solver out of the hood as it does
for dense matrices."


Cheers,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210521/899fc147/attachment.html>