[Numpy-discussion] Insights / lessons learned from NumPy design

[Chris Barker - NOAA Federal]

On Wed, Jan 9, 2013 at 2:57 AM, Mike Anderson

> I'm hoping the API will be independent of storage format - i.e. the
> underlying implementations can store the data any way they like. So the API
> will be written in terms of abstractions, and the user will have the choice
> of whatever concrete implementation best fits the specific needs. Sparse
> matrices, tiled matrices etc. should all be possible options.

A note about that -- as I think if it, numpy arrays are two things:

1) a python object for working with numbers, in a wide variety of ways

2) a wrapper around a C-array (or data block) that can be used to
provide an easyway for Python to interact with C (and Fortran, and...)
libraries, etc.

As it turns out a LOT of people use numpy for (2) -- what this means
is that while you could change the underlying data representation,
etc, and keep the same Python API -- such changes would break a lot of
non-pure-python code that relies on that data representation.

This is a big issue with the numpy-for-PyPy project -- they could
write a numpy clone, but it would only be useful for the pure-python
stuff.

Even then, a number of folks do tricks with numpy arrays in python
that rely on the underlying structure.

Not sure how all this would play out for Clojure, but it's something
to keep in mind.

-Chris



-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov