[Numpy-discussion] Introduction

[Perry Greenfield]

Hi Scott,

Just to be to the point, I'm still missing what I've been
asking for, to wit a concrete example that illustrates your
point. I'll try to address a few of your points that appear
to try to answer that and clarify what I mean by concrete
example.
>
> Here's what I'm proposing, and it's only a suggestion.
>
>
> *** I think the requirements for being a general purpose "NDArray"
> can be specified with only the following attributes:
>
>     __array_buffer__    - as buffer object
>     __array_shape__     - as tuple of long
>     __array_itemsize__  - as int
>
>     Optionally
>     __array_stride__    - as tuple of long (get from shape if None)
>     __array_offset__    - as int (would default to 0 if not present)
>
> Then anyone who implemented these could work with the same C API for
> getting the pointer to memory, shape array, stride array, and item size.
>
Then you are talking about standardizing a C-API. But I'm still
confused. If you write a class that implements these attributes,
is it your C-API that uses them, or do you mean our C-API uses
them? If you have your own C-API, then the attributes are not
relevant as an interface. If you intend to use our C-API to access
your objects, then they are. But if you want to use our C-API,
that still doesn't explain why the alternatives aren't acceptable
(namely subclassing).

>
> Because truthfully arrays are little more than a pointer to memory.
>
> That's like asking "why in the world would we presume memcpy() or
> qsort() would know what to do with your memory?"
>
Then you misunderstand Numarray. Numarrays are far more than just
a pointer to memory. You can get a pointer to memory from them,
but they entail much more than that. Numarray presumes that certain
things are possible with NumArray objects (like standard math
operations). If you want something that doesn't make such an
assumption, you should be using NDArray instead. NDArray makes
no presumptions about the contents of the memory other than
they are arranged in memory in array fashion.
>
> >
> > You haven't provided any example (let
> > alone a compelling one) of why we should accept any object that
> > provides those attributes.
> >
>
> Well, the UFuncs certainly should reject any object that they don't
> know how to handle.  I'm currently only addressing what it takes to be
> an NDArray/NumArray object.  OTOH, if I can present something to the
> UFuncs that looks like a known array type, why wouldn't UFuncs
> want to work with it?
>
If you are presenting numarray with a type is already knows about,
why aren't you subclassing it? If you present numarray an object
with a type it doesn't know about, then that is pointless.
Types and numarray are inextricably intertwined, and shall
remain so.
>
> - Allows me personally to distribute a separate (and simpler)
> implementation of NDArrays/NumArrays right now and have the same data
> objects work with yours when you're all done.  If I give the UFuncs a
> pointer to memory, and the attributes above, why shouldn't it work
> correctly?
>
>
> Am I doing any better?  I am trying.
>
Not really. More on that later.
>
>
> Is there a way, today, without modifying numarray, for me to use
> numarray as a holder for these esoteric data types?  Is that way
> difficult?
>  Could it be easier?
>
No to the first, it isn't intended to serve that purpose. If
you just need something to blindly hold values without doing
anything with them use NDArray (and you can add whatever customization
you wish regarding what methods or operators are available).

> I'm not asking numarray to know about my types in it's core baseline.  I'm
> wondering what it takes to implement new types at all.
>
It's possible to extend (but not in any way that makes it
automaticaly usable with anyone elses extension. Currently
that sort of extension would not be hard for someone that
knows how things work. We haven't documented how to do so,
and won't for a while. It's not a high priority for us now.

**********************************************************

What I want to see is a specific example. I'm not going to
pay much attention to generalities becasue I'm still unclear
about how you intend to do what you say you will do. Perhaps
I'm slow, but I still don't get it.

On the one hand, you ask us to have numarray accept objects
with the same 'interface'. Well, if they are not of an existing
supported type, thats pointless since numarray won't work
properly with them. If it is an existing type, you haven't
explained why you can't use numarray directly (or alternatively,
create a numarray object that uses the same buffer yours does).
I still haven't seen a specific example that illustrates why
you cannot use subclassing or an instance of a numarray object
instead. If you need to add a new type that's possible but
you'll have to spend some time figuring out how to do that for
your own extended version. If you just want to use arrays
to hold values (of new types), then use NDArray. It doesn't
care about types. But please give a specific case. E.g., "I want
complex ints and I will develop a class that will use this to
do the following things [it doesn't have to be exhastive or
complete, but include just enough to illustrate the point].
If the attributes were standardized then I would do this and that,
and use it with your stuff like this showing you the code
(and the behavior I expect)."

Given this I can either show you an alternate solution or
I can realize why you are right and we can discuss where
to go from there. Otherwise you are wasting your time.

Perry