Memory error with quadratic interpolation

[Isaac Won]

On Wednesday, January 23, 2013 10:51:43 AM UTC-6, Oscar Benjamin wrote:
> On 23 January 2013 14:57, Isaac Won <winefrog at gmail.com> wrote:
> 
> > On Wednesday, January 23, 2013 8:40:54 AM UTC-6, Oscar Benjamin wrote:
> 
> >> On 23 January 2013 14:28, Isaac Won <winefrog at gmail.com> wrote:
> 
> >>
> 
> [SNIP]
> 
> >
> 
> > Following is full error message after I adjusted following Ulich's advice:
> 
> >
> 
> > interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
> 
> > File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__
> 
> >      self._spline = splmake(x,oriented_y,order=order)
> 
> >   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake
> 
> >      coefs = func(xk, yk, order, conds, B)
> 
> >   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest
> 
> >      u,s,vh = np.dual.svd(B)
> 
> >   File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd
> 
> >      full_matrices=full_matrices, overwrite_a = overwrite_a)
> 
> > MemoryError
> 
> 
> 
> Where is the new code? You should show full working code (with the
> 
> import statements) and the full error that is generated by exactly
> 
> that code. If possible you should also write code that someone else
> 
> could run even without having access to your data files. If you did
> 
> that in your first post, you'd probably have an answer to your problem
> 
> by now.
> 
> 
> 
> Here is a version of your code that many people on this list can test
> 
> straight away:
> 
> 
> 
> import numpy as np
> 
> from scipy.interpolate import interp1d
> 
> x = np.array(31747 * [0.0], float)
> 
> indices = np.arange(len(x))
> 
> interp = interp1d(indices, x, kind='quadratic')
> 
> 
> 
> Running this gives the following error:
> 
> 
> 
> ~$ python tmp.py
> 
> Traceback (most recent call last):
> 
>   File "tmp.py", line 5, in <module>
> 
>     interp = interp1d(indices, x, kind='quadratic')
> 
>   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
> 
> line 308, in __init__
> 
>     self._spline = splmake(x,oriented_y,order=order)
> 
>   File "/usr/lib/python2.7/dist-packages/scipy/interpolate/interpolate.py",
> 
> line 805, in splmake
> 
>     B = _fitpack._bsplmat(order, xk)
> 
> MemoryError
> 
> 
> 
> Unless I've misunderstood how this function is supposed to be used, it
> 
> just doesn't really seem to work for arrays of much more than a few
> 
> hundred elements.
> 
> 
> 
> 
> 
> Oscar

Thank you Oscar for your help and advice.

I agree with you. So, I tried to find the way to solve this problem.

My full code adjusted is:
from scipy.interpolate import interp1d

import numpy as np
import matplotlib.pyplot as plt



with open(filin, "r") as f:

        for line in f:
                columns = line.strip().split()

                a.append(columns[5])
                x = np.array(a, float)


not_nan = np.logical_not(np.isnan(x))
indices = np.arange(len(x))
interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')

p = interp(indices)




k = np.arange(31747)

plt.subplot(211)
plt.plot(k, p)
plt.xlabel('Quadratic interpolation')
plt.subplot(212)
plt.plot(k, x)

plt.show()
-----------------------------------------------------------------
Whole error message was:

Traceback (most recent call last):
  File "QI1.py", line 22, in <module>
    interp = interp1d(indices[not_nan], x[not_nan], kind = 'quadratic')
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 311, in __init__
    self._spline = splmake(x,oriented_y,order=order)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 809, in splmake
    coefs = func(xk, yk, order, conds, B)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/interpolate/interpolate.py", line 530, in _find_smoothest
    u,s,vh = np.dual.svd(B)
  File "/lustre/work/apps/python-2.7.1/lib/python2.7/site-packages/scipy/linalg/decomp_svd.py", line 91, in svd
    full_matrices=full_matrices, overwrite_a = overwrite_a)
MemoryError
----------------------------------------------------------------------
Thank you again Oscar,

Isaac