[Numpy-discussion] 3D interpolation of large array

[Gael Varoquaux]

On Sun, Sep 13, 2009 at 11:55:43PM -0400, David Warde-Farley wrote:
> On 13-Sep-09, at 8:51 PM, brechmos wrote:


> > I have a large dataset (e.g., 70 x 500 x 500) and want to  
> > interpolate points
> > (for example to double the size).  What it seems I want is:

> > []: newx,newy,newz=mgrid[1:70:0.5,1:500:0.5,1:500:0.5]
> > []: coords = array([newz, newy, newx])
> > []: dout = np.map_coordinates(d, coords)

> That's going to incur over 3 gigabytes of RAM (about a gig each for  
> newx, newy, and newz), and you're running 32-bit Python. Not only do  
> you likely not have enough physical memory (which can be solved  
> somewhat by swapping), Python doesn't have enough memory address  
> space. So in other words, no.

> Compiling a 64-bit version of Python and NumPy would solve the problem.

Breaking the problem in smaller problems might also be an option.

Gaël