[SciPy-dev] Blitz++, NumArray...

[Perry Greenfield]

> Behalf Of Fernando Perez
> Subject: [SciPy-dev] Blitz++, NumArray...
>
[...]
> as some of you know, I'm a big fan of weave.inline() for the ease of
> development it offers.  Essentially C++ becomes a scripting part
> of python,
> with only the occasional recompile overhead (but this is
> transparent to the
> users).
>
> inline() makes heavy use of the blitz++ library, to represent
> numpy arrays as
> blitz arrays and therefore index them as Arr(i,j,k...).  This
> seems to me like
> a very sound decision, as the blitz library was specifically written to
> provide simultaneously fortran-like performance with high-level object
> orientation.
>
> With this in mind, I have a few questions:
>
> - Is there anything in Numarray's design which prevents this same
> trick from
> being used?  The core of getting a Numpy array into a blitz one
> is simply a
> call to:
>
Unfortunately, I'm neither knowledgeable (yet, anyway) about
blitz++ nor am I a C++ expert. On the other hand, with some
caveats, I would be very surprised if it isn't nearly as easy to do
the same in numarray as in numpy. Most of the C-API is identical so
it is possible that very few changes are necessary. On the other
hand, we aren't yet at the point of trying to deal with weave so
it would be a little while before we had time to tackle that
issue (at the moment, we are porting the MA package). After our
1.0 release, getting scipy to work with numarray becomes one of
our top priorities.

As for caveats, the primary one is that numarray allows some
flexibility in array storage that numpy doesn't. In particular
it allows data that is byteswapped from the normal processor order,
and arrays that are not aligned with the processor's address (both
to facilitate accessing memory mapped data and arrays of records).
For such arrays it is likely that weave will require a copy so
that the array elements may be accessed in the usual way. Since
such arrays are not permitted in numpy, this is in no way a
narrowing of capabilities.

[...]
> - If any of this seems worthy of more detailed discussion, I told
> Eric Jones
> that I'd be happy to coordinate something along these lines at
> Scipy'03.  If
> there's time in the schedule I'd be glad to do it officially,
> otherwise we can
> just try to keep it in mind for a lunch/dinner meeting.
>
Sure, but let us know what issues you think need to be worked out
in light of what I've already said. This may be a good opportunity
to get all those knowledgeable about the various components (Blitz++,
weave, and numarray) together to discuss what is needed.

> Documenting a 'Best Practices for low-level numerical work in
> python' would
> probably be a beneficial thing to do for the community.  Right
> now there are
> so many options that I'm sure I'm not the only one to feel a bit confused.
>
That's assuming that we know them all already :-) For our part we have been
concentrating on adding all the core numpy functionality and haven't yet
been able to address all issue. Over the next few months we will be
moving more into the scipy area and will be able to understand more of
the wider issues.

Perry