[Python-Dev] What's up with PEP 209: Adding Multidimensional Arrays

[Paul Barrett]

What is the status of PEP 209?  I see David Ascher is the champion of
this PEP, but nothing has been written up.  Is the intention of this
PEP to make the current Numeric a built-in feature of Python or to
re-implement and replace the current Numeric module?

The reason that I ask these questions is because I'm working on a
prototype of a new N-dimensional Array module which I call Numeric 2.
This new module will be much more extensible than the current Numeric.
For example, new array types and universal functions can be loaded or
imported on demand.  We also intend to implement a record (or
C-structure) type, because 1-D arrays or lists of records are a common
data structure for storing photon events in astronomy and related
fields.

The current Numeric does not handle record types efficiently,
particularly when the data type is not aligned and is in non-native
endian format.  To handle such data, temporary arrays must be created
and alignment and byte-swapping done on them.  Numeric 2 does such
pre- and post-processing inside the inner-most loop which is more
efficient in both time and memory.  It also does type conversion at
this level which is consistent with that proposed for PEP 208.

Since many scientific users would like direct access to the array data
via C pointers, we have investigated using the buffer object.  We have
not had much success with it, because of its implementation.  I have
scanned the python-dev mailing list for discussions of this issue and
found that it now appears to be deprecated.

My opinion on this is that a new _fundamental_ built-in type should be
created for memory allocation with features and an interface similar
to the _mmap_ object.  I'll call this a _malloc_ object.  This would
allow Numeric 2 to use either object interchangeably depending on the
circumstance.  The _string_ type could also benefit from this new
object by using a read-only version of it.  Since its an object, it's
memory area should be safe from inadvertent deletion.

Because of these and other new features in Numeric 2, I have a keen
interest in the status of PEPs 207, 208, 211, 225, and 228; and also
in the proposed buffer object.  

I'm willing to implement this new _malloc_ object if members of the
python-dev list are in agreement.  Actually I see no alternative,
given the current design of Numeric 2, since the Array class will
initially be written completely in Python and will need a mutable
memory buffer, while the _string_ type is meant to be a read-only
object.

All comments welcome.

 -- Paul

-- 
Dr. Paul Barrett       Space Telescope Science Institute
Phone: 410-338-4475    ESS/Science Software Group
FAX:   410-338-4767    Baltimore, MD 21218