Memory error with quadratic interpolation

[Isaac Won]

On Wednesday, January 23, 2013 2:55:14 AM UTC-6, Ulrich Eckhardt wrote:
> Am 23.01.2013 05:06, schrieb Isaac Won:
> 
> > I have tried to use different interpolation methods with Scipy. My
> 
> > code seems just fine with linear interpolation, but shows memory
> 
> > error with quadratic. I am a novice for python. I will appreciate any
> 
> > help.
> 
>  >
> 
> > #code
> 
> > f = open(filin, "r")
> 
> 
> 
> Check out the "with open(...) as f" syntax.
> 
> 
> 
> 
> 
> > for columns in ( raw.strip().split() for raw in f ):
> 
> 
> 
> For the record, this first builds a sequence and then iterates over that 
> 
> sequence. This is not very memory-efficient, try this instead:
> 
> 
> 
>     for line in f:
> 
>         columns = line.strip().split()
> 
> 
> 
> 
> 
> Concerning the rest of your problems, there is lots of code and the 
> 
> datafile missing. However, there is also too much of it, try replacing 
> 
> the file with generated data and remove everything from the code that is 
> 
> not absolutely necessary.
> 
> 
> 
> Good luck!
> 
> 
> 
> Uli

Hi Ulich,

I tried to change the code following your advice, but it doesn't seem to work still.

My adjusted code is:

a = []

with open(filin, "r") as f:

        for line in f:
                columns = line.strip().split()

                a.append(columns[5])
                x = np.array(a, float)


not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
p = interp(indices)
---------------------------------------------------------------------
And full error message is:
   interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__
    self._spline = splmake(x,oriented_y,order=order)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake
    coefs = func(xk, yk, order, conds, B)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest
    u,s,vh = np.dual.svd(B)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd
    full_matrices=full_matrices, overwrite_a = overwrite_a)
MemoryError
-----------------------------------------------------------------------
Could you give me some advice for this situation?

Thank you always,

Isaac