[Numpy-discussion] The array interface published

[Magnus Lie Hetland]

Michiel Jan Laurens de Hoon <mdehoon at ims.u-tokyo.ac.jp>:
>
[snip]
> 1) The "__array_shape__" method is identical to the existing "shape" method 
> in Numerical Python and numarray (except that "shape" does a little bit 
> better checking, but it can be added easily to "__array_shape__"). To avoid 
> code duplication, it might be better to keep that method. (and rename the 
> other methods for consistency, if desired).

Why not just use 'shape' as an alias for '__array_shape__' (or vice
versa)?

> 2) The __array_datalen__ is introduced to get around the 32-bit int
> limitation of len(). Another option is to fix len() in Python
> itself, so that it can return integers larger than 32 bits. So we
> can avoid adding a new method.

That would bee good, IMO. But how realistic is it? (I have no idea --
this is not a rhetorical question :)

> 3) Where do default values come from? Is it the responsability of the 
> extension module writer to find out if the array module implements e.g. 
> __array_strides__, and substitute the default values if it doesn't?

If the support of these attributes is optional, that would have to be
the case.

> If so, I have a slight preference to make all methods required,
> since it's not a big effort to return the defaults, and there will
> be more extension modules than array packages (or so I hope).

But isn't the point that you should be able to export other things
(such as images or sounds or what-have-you) *as* arrays?

As for implementing the defaults: How about having some utility
functions (or a wrapper object or whatever) that does just this -- so
neither array nor client code need think about it? This could,
perhaps, be put in the stdlib array module or something...

> Whereas the array interface certainly helps extension writers to
> create an extension module that works with all array
> implementations, it also enables and perhaps encourages the creation
> of different array modules, while our original goal was to create a
> single array module that satisfies the needs of both Numerical
> Python and numarray users. I still think such a solution would be
> preferable.

I agree.

But what I think would be cool if such a standardized package could
take any object conforming to this protocol and use it (possibly as
the argument to the array() constructor) -- with all the ufuncs and
operators it has. Because then I could implement specialized arrays
where the specialized behaviour lies just in the data itself, not the
behaviour. For example, I might want to create a thin array wrapper
around a memory-mapped, compressed video file, and treat it as a
three-dimensional array of rgb triples... (And so forth.)

> Inconsistencies other than the array interface (e.g.  one implements
> argmax(x) while another implements x.argmax()) may mean that an
> extension module can work with one array implementation but not with
> another,

This does *not* sound like a good thing -- I agree. Certainly not what
I would hope this protocol is used for.

> even though they both conform to the array interface. We may end up
> with several array packages (we already have Numerical Python,
> numarray, and scipy), and extension modules that work with one
> package and not with another. So in a sense, the array interface is
> letting the genie out of the bottle.

Well, perhaps -- but the current APIs of e.g., Numeric or numarray
could be used in the same way (i.e., writing your own array
implementations with the same interface).

As (I think) Travis has said, there is still a goal (somewhat separate
from the protocol) of getting one standard heavy-duty numerical array
package. I think that would be very beneficial. The point (as I see
it) is just to make it easier for various array implementations (i.e.,
the data, not the ufuncs/operators etc.) to interoperate with it.

> But maybe such a single array package is not attainable given the
> different needs of the different communities.

I would certainly hope it is.

> --Michiel.

-- 
Magnus Lie Hetland                    Fall seven times, stand up eight
http://hetland.org                                  [Japanese proverb]