[SciPy-User] matplotlib and large array

[David Baddeley]

Hi Massimo,

matplotlib is pretty awful with huge images - part of the problem is that if stores a colormapped copy of the image which means that you go from a single image to R,G,B, & A channels. I have a feeling that this copy is also using a floating point datatype. In any case you're multiplying your memory usage by at least 4 (potentially much more if it is using floats/doubles and your data is 8 or 16 bit integers). Another problem is that it interpolates when the screen pixels and image pixels don't match up (which is most of the time). This is really nice when you've got small images you're scaling up, but quite a performance drag for larger images.

I ended up coding up my own, wxpython based, viewer which downsamples at low magnification and only pulls out the currently visible ROI at high mag. Only this visible region is then colourmapped (at most the window size - so around 1k by 1k pixels if you've got the window maximised). This makes a huge difference to memory consumption, and performance. There's no provision for plotting axes or other stuff over the image though. My viewer is designed for 3D images with multiple colour channels (microscopy data sets), but will handle 2D images fine. Unfortunately it's currently got a fair bit of application dependent cruft/dependencies. I've been meaning to strip these out so it can be used standalone for a while, so if you don't find an alternative viewer, drop me a line and I'll see if I can get it into some sort of shape.

cheers,
David


----- Original Message ----
From: Massimo Di Stefano <massimodisasha at gmail.com>
To: matplotlib-users at lists.sourceforge.net
Cc: SciPy Users List <scipy-user at scipy.org>
Sent: Thu, 10 June, 2010 6:31:17 AM
Subject: [SciPy-User] matplotlib and large array

Hi All,

i need to work with a relative large images "60 mb" (single band geotiff file)
i store it in python as a numpy array using python-gdal, 
the array dinension is (7173 X 7924) single band image,
but tring to display it with matshow/imageshow
or other matplotlib functions i have that python freeze itself and is not able to load the image.

if i use a subset of the image, i 'm able to display it
or at least i hade to reduce its resolution using hacks like :

reduced_array = array[::3,::3]

i don't need full resolution dataset when the image is displaied with a full "zoom out"
so the reduction " reduced_array = array[::3,::3] "  is good to show the complete image
but when i zoom in the image i obviously lost data (less resolution)

what do you use to display large dataset ?

i'm thinking  about a "piramid" with multy array based on the different zoom levels
.. but maybe this idea is not so cool.
someone already has developed similar code ?

thanks to All for any suggestion!

Regards,

Massimo
_______________________________________________
SciPy-User mailing list
SciPy-User at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-user