[Numpy-discussion] A comparative test

[Fernando Perez]

Rick White wrote:
> IMHO the numarray developers put their emphasis in the right place
> by focusing on large array performance and improved functionality,
> and all the noise around small array indexing performance was just
> a red herring that convinced some folks not to try numarray because
> they heard it was slow.  I hope Travis doesn't devote a lot of
> effort to this optimization right now.  I'd be much more interested
> in seeing a large array benchmark.

Except for codes like our PDE solvers, which need to create 10s of millions of 
small arrays very, very fast.  So no, it's not a red herring [1].  Just 
because you don't need something doesn't mean there's no valid need for it.

I am not disputing in any way the value of the many, many improvements made by 
numarray, nor the importance of fixing large array performance for many 
applications (such as I imagine is the typical workflow of astronomical data 
analysis).  The fact that Travis tried to incorporate all of numarrays' 
improvements into the new library is a recognition of the value of all this work.

But calling the small array performance problem a 'red herring' is inaccurate 
and unfair, at best.

Regards,

f


[1] Yes, I could preallocate pools of memory for this, manage them manually, 
etc.  But then I wouldn't be writing my code in python, would I?  The whole 
point of using python is to have my cake and eat it too: I want high-level 
code that gives me near-hardware max speeds.  With carefully tuned (python) 
code, judiciously sprinkled minimal amounts of C/C++/Fortran, and a LOT of 
thinking about algorithm design, I get that.