[Numpy-discussion] Fortran order arrays to and from numpy arrays

[Alexander Schmolck]

Andrew Straw <strawman at astraw.com> writes:

> Alexander Schmolck wrote:
> > 2. Despite this overhead, copying around large arrays (e.g. >=1e5 elements) in
> >    above way causes notable additional overhead. Whilst I don't think there's
> >    a sane way to avoid copying by sharing data between numpy and matlab the
> >    copying could likely be done better.
> >
> 
> Alex, what do you think about "hybrid arrays"?
> 
> http://www.mail-archive.com/numpy-discussion@lists.sourceforge.net/msg03748.html

Oh, that's why I said no *sane* way :) 

I read about hybrid arrays, but as far as I can tell the only way to use them
to avoid copying stuff around is to create your own hybrid array memory pool
(as you suggested downthread), which seems a highly unattractive pain-for-gain
trade-off, especially given that you *do* have to reorder the data as you go
from numpy to matlab -- unless of course I'm missing something. I have the
impression that just memcpy'ing a single chunk of data isn't that much of a
performance sink, but the reordering copying that mlabwrap currently does
seems rather expensive.

In other words, I think going from matlab to numpy is fine (just memcpy into a
newly created fortran-order numpy array, or more or less equivalently, memcpy
into a C-order array, transpose and reshape), the question appears to be how
to best go from numpy to matlab when the numpy array isn't fortran-contiguous.
I assume that it makes more sense to rely on some numpy functionality than to
use a custom reordering-copy routine, especially if I want to move to ctypes
later. Is there anything better than

1. allocating a matlab array
2. transposing and reshaping the numpy array
3. allocating (or keeping around) a temporary numpy array with data
   pointing to the matlab array data
4. using some function (PyArray_CopyInto?) to copy from the transposed,
   reshaped numpy array into the temporary numpy array thereby filling the
   matlab array with an appropriately reordered copy of the original array

?

cheers,

alex