[Numpy-discussion] Numarray header PEP

[gerard.vermeulen at grenoble.cnrs.fr]

On 30 Jun 2004 17:54:19 -0400, Todd Miller wrote
> 
> So... you use the "meta" code to provide package specific ordinary
> (not-macro-fied) functions to keep the different versions of the
> Present() and isArray() macros from conflicting.
> 
> It would be nice to have a standard approach for using the same
> "extension enhancement code" for both numarray and Numeric.  The PEP
> should really be expanded to provide an example of dual support for one
> complete and real function, guts and all, so people can see the process
> end-to-end;  Something like a simple arrayprint.  That process needs 
> to be refined to remove as much tedium and duplication of effort as 
> possible.  The idea is to make it as close to providing one 
> implementation to support both array packages as possible.  I think it's
> important to illustrate how to partition the extension module into
> separate compilation units which correctly navigate the dual
> implementation mine field in the easiest possible way.
> 
> It would also be nice to add some logic to the meta-functions so that
> which array package gets used is configurable.  We did something like
> that for the matplotlib plotting software at the Python level with 
> the "numerix" layer, an idea I think we copied from Chaco.  The kind 
> of dispatch I think might be good to support configurability looks like
> this:
> 
> PyObject *
> whatsThis(PyObject *dummy, PyObject *args)
> {
>     PyObject *result, *what = NULL;
>     if (!PyArg_ParseTuple(args, "O", &what))
>       return 0;
>     switch(PyArray_Which(what)) {
>       USE_NUMERIC:
>          result = Numeric_whatsThis(what); break;
>       USE_NUMARRAY:
>          result = Numarray_whatsThis(what); break;
>       USE_SEQUENCE:
>          result = Sequence_whatsThis(what); break;
>     }
>     Py_INCREF(Py_None);
>     return Py_None;
> }
> 
> In the above,  I'm picturing a separate .c file for Numeric_whatsThis
> and for Numarray_whatsThis.  It would be nice to streamline that to one
> .c and a process which somehow (simply) produces both functions.
> 
> Or, ideally, the above would be done more like this:
> 
> PyObject *
> whatsThis(PyObject *dummy, PyObject *args)
> {
>     PyObject *result, *what = NULL;
>     if (!PyArg_ParseTuple(args, "O", &what))
>        return 0;
>     switch(Numerix_Which(what)) {
>        USE_NUMERIX:
>           result = Numerix_whatsThis(what); break;
>        USE_SEQUENCE:
>           result = Sequence_whatsThis(what); break;
>     }
>     Py_INCREF(Py_None);
>     return Py_None;
> }
> 
> Here, a common Numerix implementation supports both numarray and Numeric
> from a single simple .c.  The extension module would do "#include
> numerix/arrayobject.h" and "import_numerix()" and otherwise just call
> PyArray_* functions.
> 
> The current stumbling block is that numarray is not binary compatible
> with Numeric... so numerix in C falls apart.  I haven't analyzed 
> every symbol and struct to see if it is really feasible... but it 
> seems like it is *almost* feasible, at least for typical usage.
> 
> So, in a nutshell,  I think the dual implementation support you 
> demoed is important and we should work up an example and kick it 
> around to make sure it's the best way we can think of doing it.  
> Then we should add a section to the PEP describing dual support as well.
> 
I would never apply numarray code to Numeric arrays and the inverse. It looks
dangerous and I do not know if it is possible.  The first thing coming
to mind is that numarray and Numeric arrays refer to different type objects
(this is what my pep module uses to differentiate them).  So, even if
numarray and Numeric are binary compatible, any 'alien' code referring the
the 'Python-standard part' of the type objects may lead to surprises.
A PEP proposing hacks will raise eyebrows at least. 

Secondly, most people use Numeric *or* numarray and not both.

So, I prefer: Numeric In => Numeric Out or Numarray In => Numarray Out (NINO)
Of course, Numeric or numarray output can be a user option if NINO does not
apply.  (explicit safe conversion between Numeric and numarray is possible
if really needed).

I'll try to flesh out the demo with real functions in the way you indicated
(going as far as I consider safe).

The problem of coding the Numeric (or numarray) functions in more than
a single source file has also be addressed.

It may take 2 weeks because I am off to a conference next week.

Regards -- Gerard