[Numpy-discussion] Any Numeric or numarray users on this list?

[Fernando Perez]

On 5/30/06, Travis Oliphant <oliphant.travis at ieee.org> wrote:
>
> Please help the developers by responding to a few questions.

Sorry for not replying before, I wanted a more complete picture before
answering.

> 1) Have you transitioned or started to transition to NumPy (i.e. import
> numpy)?

The day this email came in, I had just started to look into porting
our major research code.  I actually did the work 2 weeks ago, and it
went remarkably well.  It took a single (marathon) day, about 14 hours
of solid work, to go through the old codebase and convert it.  This
project had a mix of legacy Fortran wrapped via f2py, hand-written C
extensions using Numeric, a fair bit of weave.inline() and pure
python.  It uses matplotlib, PyX and Mayavi for various visualization
tasks.

There are some 40k loc in the Fortran sources (2/3 of that
auto-generated in python from Mathematica computations), and about 13k
loc in the C and python sources.

This codebase is heavily unit-tested, which was critical for the port.
 For this kind of effort, unittests make an enormous difference, as
they guide you directly to all the problematic spots.  Without
unittests, this kind of port would have been a nightmare, and I would
have never known whether things were actually finished or not.

Most of my changes had to do with explicit uses of 'typecode=' which
became dtype, and uses of .flat, which used to return a normal array
and is now an iterator.

I haven't benchmarked things right away, because I expect the
numpy-based code to take quite a hit.  In this code, I've heavily
abused arrays for very trivial 2 and 3-element arithmetic operations,
but that means that I create literally millions of extremely small
arrays.  Even with Numeric, this overhead was already measurable, and
I imagine it will get worse with numpy.  But since this was silly
anyway, and I need to use these little arrays as dictionary keys,
instead of doing lots of tuple(array()) all the time, I'm using David
Cooke's Vector as a template for a hand-written mini-array class that
will do exactly what I need with as little overhead as possible.

If for any reason you do want to see actual benchmarks, I can try to
run some with the codebases immediately before and after the
Numeric->numpy change and report back.

> 2) Will you transition within the next 6 months? (if you answered No to #1)

That's it: by now we've moved all of our code and it doesn't really
work with Numeric anymore, so we're committed :)


Again, many thanks for the major improvements that numpy brings!

Cheers,

f