numarray speed question

[David M. Cooke]

At some point, grv575 at hotmail.com (grv575) wrote:

>> What you *do* get with numarray is:
>> 
>> 1) transparent handling of byteswapped, misaligned, discontiguous,
>>    type-mismatched data (say, from a memory-mapped file generated on a
>>    system with a different byte order as single-precision instead of
>>    double-precision).
>
> Heh.  Try timing the example I gave (a += 5) using byteswapped vs.
> byteswap().  It's fairly fast to do the byteswap.  If you go the
> interpretation way (byteswapped) then all subsequent array operations
> are at least an order of magnitude slower (5 million elements test
> example).

You mean something like
a = arange(0, 5000000, type=Float64).byteswapped()
a += 5

vs.
a = arange(0, 5000000, type=Float64)
a.byteswap()
a += 5

? I get the same time for the a+=5 in each case -- and it's only twice
as slow as operating on a non-byteswapped version. Note that numarray
calls the ufunc add routine with non-byteswapped numbers; it takes a
block, orders it correctly, then adds 5 to that, does the byteswap on
the result, and stores that back. (You're not making a full copy of
the array; just a large enough section at a time to do useful work.)

>> If you need the best possible speed (after doing it in numarray and
>> finding it isn't fast enough), you can write an extension module to
>> do that bit in C, or look into scipy.weave for inlining C code, or into
>> f2py for linking Fortran code to Python.
>
> Well re speed what really bothers me is the slowness in which numarray
> is improving in this area.  If I have to take 1000 FFT's over 32
> element arrays, then it's useless.  I'll have to install both numarray
> and numeric :/

Maybe what you need is a package designed for *small* arrays ( < 1000).
Simple C wrappers; just C doubles and ints, no byteswap, non-aligned.
Maybe a fixed number of dimensions. Probably easy to throw something
together using Pyrex. Or, wrap blitz++ with boost::python.

-- 
|>|\/|<
/--------------------------------------------------------------------------\
|David M. Cooke
|cookedm(at)physics(dot)mcmaster(dot)ca