[SciPy-Dev] Enhancement Request: SuperLU_dist/SuperLU_mt replace current scipy.sparse.superLU

[Neil Godber]

Hi dev-list,

Further last, realised that I failed to link to the issue itself despite
linking to virtually everything else:

Replace superLU sequential with superLU dist · Issue #14096 · scipy/scipy
(github.com) <https://github.com/scipy/scipy/issues/14096>

Cheers,
Neil

On Fri, 21 May 2021 at 10:04, Neil Godber <njgodber at gmail.com> wrote:

> Hi dev-list,
>
> I've logged an enhancement request on github to bring the
> parallel versions of SuperLU into scipy. As requested I'm circulating the
> enhancement here to solicit comment/feedback/engagement.
>
> Content of request is as follows (with light editing and links):
>
> "Is your feature request related to a problem? Please describe.
>
> The direct solution of sparse matrices is a common problem that arises
> across many domains. Current scipy.sparse.linalg provides spsolve and splu
> to solve such classes of problems. Neither solution appears to utilise more
> than one or two threads leaving a lot of the available compute on modern
> machines unused. Other solutions (MUMPS, Pardiso) exploit multicore
> processors but are either minimally supported and/or locked behind
> proprietary libraries.
>
> Describe the solution you'd like
> Fortunately SuperLU has two multithreaded implementations superLU_mt and
> superLU_dt, both actively maintained (SuperLU: Home Page (nersc.gov)
> <https://portal.nersc.gov/project/sparse/superlu/>). Ideally SciPy would
> replace the current sequential implementation with one of the parallel
> implementations to yield large performance improvements.
>
> Describe alternatives you've considered
> pyMKL (Pardiso dwfmarchant/pyMKL: Python wrappers to Intel MKL routines
> (github.com) <https://github.com/dwfmarchant/pyMKL>) - MKL with issues
> associated with that for non Intel processors, unmaintained relatively to
> scipy, PyTrilinos (implements all three superLU types but only available on
> osx and linux PyTrilinos | Trilinos
> <https://trilinos.github.io/pytrilinos.html>), petsc (petsc4py PETSc /
> petsc · GitLab <https://gitlab.com/petsc/petsc>) (implements all three
> superLU types but only available on osx and linux). All are relatively
> inaccessible compared to ubiquity and accessibility of scipy.
>
> Additional context (e.g. screenshots)
> Anecdotally large number of projects use various non scipy sparse solvers
> to achieve high level of performance. This comes with considerable time and
> maintenance investment. It would be fantastic for scipy to provide a
> 'performance competitive' sparse direct solver out of the hood as it does
> for dense matrices."
>
>
> Cheers,
>
> Neil
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210521/b109a7a6/attachment-0001.html>