[Numpy-discussion] lazy loading ndarrays

[Matthew Brett]

Hi,

On Tue, Jul 26, 2011 at 5:11 PM, Craig Yoshioka <craigyk at me.com> wrote:
> I want to subclass ndarray to create a class for image and volume data, and when referencing a file I'd like to have it load the data only when accessed.  That way the class can be used to quickly set and manipulate header values, and won't load data unless necessary.  What is the best way to do this?  Are there any hooks I can use to load the data when an array's values are first accessed or manipulated?  I tried some trickery with __array_interface__ but couldn't get it to work very well.  Should I just use a memmapped array, and give up on a purely 'lazy' approach?

What kind of images are you loading?   We do lazy loading in nibabel,
for medical image type formats:

http://nipy.sourceforge.net/nibabel/

- but our images _have_ arrays and headers, rather than (appearing to
be) arrays.  Thus something like:

import nibabel as nib

img = nib.load('my_image.img')
# data not loaded at this point
data = img.get_data()
# data loaded now.  Maybe memmapped if the format allows

If you think you might have similar needs, I'd be very happy to help
you get going in nibabel...

Best,

Matthew