[SciPy-Dev] Minimal sparse array interface

[Stefan van der Walt]

Hi Evgeni,

On Thu, Jun 3, 2021, at 12:34, Evgeni Burovski wrote:
> You're thinking something like a lightweight `csr_array` which is
> almost identical to the existing csr_matrix, but has  a np.array-like
> multiplication?

Exactly, and that also conforms with indexing expectations.

> 1. What's the minimal subset of formats? Would CSR be enough, or some
> others are needed from the start?

I suspect we will have to cover most of them in order for this to be useful.  I just looked at NetworkX, for example, and they use CSR, LIL, and COO at least.

> 2. Should this live in scipy.sparse or keep it in a separate repo in
> the scipy org for a while?

Most of it we can probably build separately, but it may require a few tweaks to the existing classes.  Those changes could simply be PRs (once there is rough consensus that we want to go this route).

> 3.What about d != 2 ? E.g., does array[0, :] behave as a 1D numpy
> array. What about array[:, :, None]. Does it broadcast? (This can
> develop into quite a rabbit hole unless defined very clearly from the
> start I suspect.)

Indexing is probably the trickiest part to get right.  A first thought would be to limit any interactions that produce >2D, and to introduce 1d and 0d.  I'd have to double check, but it may be possible to do shape calculations, follow that by relying on existing operations to produce a result, and then adjust that result to conform to the expected shape.

> 4. The interaction with scipy.sparse.linalg.

We'd need some way to let `scipy.sparse.linalg` know what's going on.  One way would be to have `linalg` call `as_sparse_matrix` or `as_sparse_array` explicitly, depending on its needs.

Stéfan