[SciPy-user] non contiguous numpy arrays

[Travis Oliphant]

Gary S. Thompson wrote:

>Hi
>    I have read the stuff on the numpy interface and understand most of 
>if(?) ;-) However I want to interface numpy to an array which is stored 
>as a series of data blocks which are not contiguous in memory... Help!
>
>  
>

>so for example I hava 3d matrix of 1024*512*128 where data is stored in 
>blocks within memory which are 16x16x16 floats an read in from disk on 
>demand...
>
>  
>
The memory model for numpy arrays requires that you can access the next 
element in each dimension by striding a "fixed-number" of bytes.

In other words, to be a single ndarray, element i,j,k of the array, B, 
must be at

start_of_data + i*B.stride[0] + j*B.stride[1] + k*B.stride[2]

So, in your case, do these 16x16x16 blocks of floats all exist at 
arbitrary memory locations?  If so, then I don't see how you can map 
that to a single ndarray.

You could, however, write a class that wraps each block as a separate 
sub-array and do the indexing calculations yourself to determine which 
sub-block the data is stored in.

But, I don't see a way to use a single ndarray to access memory layed 
out like that.  Perhaps I still don't understand what you are doing.  
Are you using a memory map?


-Travis