[Numpy-discussion] Numarray header PEP
Todd Miller
jmiller at stsci.edu
Thu Jul 1 09:44:13 EDT 2004
On Thu, 2004-07-01 at 02:33, gerard.vermeulen at grenoble.cnrs.fr wrote:
> On 30 Jun 2004 17:54:19 -0400, Todd Miller wrote
> >
> > So... you use the "meta" code to provide package specific ordinary
> > (not-macro-fied) functions to keep the different versions of the
> > Present() and isArray() macros from conflicting.
> >
> > It would be nice to have a standard approach for using the same
> > "extension enhancement code" for both numarray and Numeric. The PEP
> > should really be expanded to provide an example of dual support for one
> > complete and real function, guts and all, so people can see the process
> > end-to-end; Something like a simple arrayprint. That process needs
> > to be refined to remove as much tedium and duplication of effort as
> > possible. The idea is to make it as close to providing one
> > implementation to support both array packages as possible. I think it's
> > important to illustrate how to partition the extension module into
> > separate compilation units which correctly navigate the dual
> > implementation mine field in the easiest possible way.
> >
> > It would also be nice to add some logic to the meta-functions so that
> > which array package gets used is configurable. We did something like
> > that for the matplotlib plotting software at the Python level with
> > the "numerix" layer, an idea I think we copied from Chaco. The kind
> > of dispatch I think might be good to support configurability looks like
> > this:
> >
> > PyObject *
> > whatsThis(PyObject *dummy, PyObject *args)
> > {
> > PyObject *result, *what = NULL;
> > if (!PyArg_ParseTuple(args, "O", &what))
> > return 0;
> > switch(PyArray_Which(what)) {
> > USE_NUMERIC:
> > result = Numeric_whatsThis(what); break;
> > USE_NUMARRAY:
> > result = Numarray_whatsThis(what); break;
> > USE_SEQUENCE:
> > result = Sequence_whatsThis(what); break;
> > }
> > Py_INCREF(Py_None);
> > return Py_None;
> > }
> >
> > In the above, I'm picturing a separate .c file for Numeric_whatsThis
> > and for Numarray_whatsThis. It would be nice to streamline that to one
> > .c and a process which somehow (simply) produces both functions.
> >
> > Or, ideally, the above would be done more like this:
> >
> > PyObject *
> > whatsThis(PyObject *dummy, PyObject *args)
> > {
> > PyObject *result, *what = NULL;
> > if (!PyArg_ParseTuple(args, "O", &what))
> > return 0;
> > switch(Numerix_Which(what)) {
> > USE_NUMERIX:
> > result = Numerix_whatsThis(what); break;
> > USE_SEQUENCE:
> > result = Sequence_whatsThis(what); break;
> > }
> > Py_INCREF(Py_None);
> > return Py_None;
> > }
> >
> > Here, a common Numerix implementation supports both numarray and Numeric
> > from a single simple .c. The extension module would do "#include
> > numerix/arrayobject.h" and "import_numerix()" and otherwise just call
> > PyArray_* functions.
> >
> > The current stumbling block is that numarray is not binary compatible
> > with Numeric... so numerix in C falls apart. I haven't analyzed
> > every symbol and struct to see if it is really feasible... but it
> > seems like it is *almost* feasible, at least for typical usage.
> >
> > So, in a nutshell, I think the dual implementation support you
> > demoed is important and we should work up an example and kick it
> > around to make sure it's the best way we can think of doing it.
> > Then we should add a section to the PEP describing dual support as well.
> >
> I would never apply numarray code to Numeric arrays and the inverse. It looks
> dangerous and I do not know if it is possible.
I think that's definitely the marching orders for now... but you gotta
admit, it would be nice.
> The first thing coming
> to mind is that numarray and Numeric arrays refer to different type objects
> (this is what my pep module uses to differentiate them). So, even if
> numarray and Numeric are binary compatible, any 'alien' code referring the
> the 'Python-standard part' of the type objects may lead to surprises.
> A PEP proposing hacks will raise eyebrows at least.
I'm a little surprised it took someone to talk me out of it... I'll
just concede that this was probably a bad idea.
> Secondly, most people use Numeric *or* numarray and not both.
A class of question which will arise for developers is this: "X works
with Numeric, but X doesn't work with numaray." The reverse also
happens occasionally. For this reason, being able to choose would be
nice for developers.
> So, I prefer: Numeric In => Numeric Out or Numarray In => Numarray Out (NINO)
> Of course, Numeric or numarray output can be a user option if NINO does not
> apply.
When I first heard it, I though NINO was a good idea, with the
limitation that it doesn't apply when a function produces an array
without consuming any. But... there is another problem with NINO that
Perry Greenfield pointed out: with multiple arguments, there can be a
mix of array types. For this reason, it makes sense to be able to
coerce all the inputs to a particular array package. This form might
look more like:
switch(PyArray_Which(<no_parameter_at_all!>)) {
case USE_NUMERIC:
result = Numeric_doit(a1, a2, a3); break;
case USE_NUMARRAY:
result = Numarray_doit(a1, a2, a3); break;
case USE_SEQUENCE:
result = Sequence_doit(a1, a2, a3); break;
}
One last thing: I think it would be useful to be able to drive the code
into sequence mode with arrays. This would enable easy benchmarking of
the performance improvement.
> (explicit safe conversion between Numeric and numarray is possible
> if really needed).
>
>I'll try to flesh out the demo with real functions in the way you indicated
> (going as far as I consider safe).
>
> The problem of coding the Numeric (or numarray) functions in more than
> a single source file has also be addressed.
>
> It may take 2 weeks because I am off to a conference next week.
Excellent. See you in a couple weeks.
Regards,
Todd
More information about the NumPy-Discussion
mailing list