[Numpy-discussion] Re: The future of Numpy

[Perry Greenfield]

> Mark Esplin wrote:
> > 
> > What is the plan for the future development of Numerical Python?  For
> > example, will additional special functions be included in numpy, or is
> > NumPy to be a building block for other applications?  I guess the big
> > question is  "Will Numerical Python become a complete computing
> > environment like Matlab or IDL?"
> > 
> > There is an interesting series of article on Numerical Python by Eric
> > Hagemann on O'Reilly network (www.oreillynet.com) where Eric Hagemann
> > is using NumPy with other python modules as a computing environment
> > more than as a software development framework.
> > 
> > I think NumPy could become a lot like the development of linux.  Linux
> > strictly speaking is only the operating system kernel.  However, a
> > linux system has come to includes a whole set of packages that work
> > together to make up a whole system.
> 
> See PEP 209: Multi-dimensional Arrays
> 
>   http://python.sourceforge.net/peps/pep-0209.html
> 
> -- 
> Paul Barrett, PhD      Space Telescope Science Institute
>

We at STScI have been working on a new implementation of Numeric. 
We are far enough along to release an incomplete version for evaluation
by the Numeric (the current version) developers in about 2 weeks.
The existing PEP (209) doesn't address all the details of the
implementation and there are some differences from the PEP, but
it does give a general idea of what our goals are. We have been
more concerned at this point in implementation than keeping the
PEP up to date. The PEP will be revised when we release a preliminary
version. The preliminary version will be

0) Called something else. Probably numarray.
1) Not completely backwards compatible with the existing Numeric.
   (I don't have time to detail the differences, but they will be
   listed when we make it availble).
2) Incomplete. It will be missing functionality, especially 
   with regard to structural operations (choose, where, nonzero, put,
   take, compress...). It will have most types (excepting complex) and
   most standard Ufuncs. We do not expect this version will be usuable
   for most projects because of this. We are looking for feedback on 
   the interface and design approach rather than advertising it as a
   usuable version. We also are interested in collaborators in
   implementing the rest of the functionality. It probably will be
   moved to sourceforge soon after.
3) Slower for smaller arrays.  We expect that performance for large
   arrays (>100,000 elements) should be fairly respectable, but since
   most of the original implementation is in Python, the overhead for
   smaller arrays is substantially increased over the existing Numeric.
   While we did benchmarking for determining what design approaches 
   were efficient enough, we have eschewed benchmarks until we have
   completed all the important functionality (soon). At that point
   we will begin to work on optimization to improve the performance.

I'll stop here, but you should be hearing more about it in the near
future.

Perry Greenfield
Space Telescope Science Institute
Science Software Group