[Numpy-discussion] numpy arrays, data allocation and SIMD alignement
David Cournapeau
david at ar.media.kyoto-u.ac.jp
Wed Aug 8 23:08:45 EDT 2007
Charles R Harris wrote:
> Anne,
>
> On 8/8/07, *Anne Archibald* <peridot.faceted at gmail.com
> <mailto:peridot.faceted at gmail.com>> wrote:
>
> On 08/08/2007, Charles R Harris <charlesr.harris at gmail.com
> <mailto:charlesr.harris at gmail.com>> wrote:
> >
> >
> > On 8/8/07, Anne Archibald <peridot.faceted at gmail.com
> <mailto:peridot.faceted at gmail.com>> wrote:
> > > Oh. Well, it's not *terrible*; it gets you an aligned array.
> But you
> > > have to allocate the original array as a 1D byte array (to
> allow for
> > > arbitrary realignments) and then align it, reshape it, and
> reinterpret
> > > it as a new type. Plus you're allocating an extra ndarray
> structure,
> > > which will live as long as the new array does; this not only
> wastes
> > > even more memory than the portable alignment solutions, it
> clogs up
> > > python's garbage collector.
> >
> > The ndarray structure doesn't take up much memory, it is the
> data that is
> > large and the data is shared between the original array and the
> slice. Nor
> > does the data type of the slice need changing, one simply uses
> the desired
> > type to begin with, or at least a type of the right size so that
> a view will
> > do the job without copies. Nor do I see how the garbage
> collector will get
> > clogged up, slices are a common feature of using numpy. The
> slice method
> > also has the advantage of being compiler and operating system
> independent,
> > there is a reason Intel used that approach.
>
I am not sure to understand which approach to which problem you are
talking about here ?
IMHO, the discussion is becoming a bit carried away. What I was
suggesting is
- being able to check whether a given data buffer is aligned to a
given alignment (easy)
- being able to request an aligned data buffer: requires aligned
memory allocators, and some additions to the API for creating arrays.
This all boils down to the following case: I have a C function which
requires N bytes aligned data, I want the numpy API to provide this
capability. I don't understand the discussion on doing it in python:
first, this means you cannot request a data buffer at the C level, and I
don't understand the whole discussion on slice, multi dimension and so
on either: at the C level, different libraries may need different arrays
formats, and in the case of fftw, all it cares about is the alignment of
the data pointer. For contiguous, C order arrays, as long as the data
pointer is aligned, I don't think we need more; are some people familiar
with the MKL, who could tell whether we need more ?
cheers,
David
More information about the NumPy-Discussion
mailing list