[Numpy-discussion] new NEP: np.AbstractArray and np.asabstractarray

[Stephan Hoyer]

On Thu, Mar 8, 2018 at 5:54 PM Juan Nunez-Iglesias <jni.soma at gmail.com>
wrote:

> On Fri, Mar 9, 2018, at 5:56 AM, Stephan Hoyer wrote:
>
> Marten's case 1: works exactly like ndarray, but stores data differently:
> parallel arrays (e.g., dask.array), sparse arrays (e.g.,
> https://github.com/pydata/sparse), hypothetical non-strided arrays (e.g.,
> always C ordered).
>
>
> Two other "hypotheticals" that would fit nicely in this space:
> - the Open Connectome folks (https://neurodata.io) proposed linearising
> indices using space-filling curves, which minimizes cache misses (or IO
> reads) for giant volumes. I believe they implemented this but can't find it
> currently.
> - the N5 format for chunked arrays on disk:
> https://github.com/saalfeldlab/n5
>

I think these fall into another important category of duck arrays.
"Indexable" arrays the serve as storage, but that don't support
computation. These sorts of arrays typically support operations like
indexing and define handful of array-like properties (e.g., dtype and
shape), but not arithmetic, reductions or reshaping.

This means you can't quite use them as a drop-in replacement for NumPy
arrays in all cases, but that's OK. In contrast, both dask.array and sparse
do aspire to do fill out nearly the full numpy.ndarray API.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20180309/9c15eb69/attachment.html>