[SciPy-user] Polyfit may be poorly conditioned

[Anne Archibald]

On 27/03/2008, massimo sandal <massimo.sandal at unibo.it> wrote:
> Anne Archibald ha scritto:
>
>  > Since you are computing polynomial coefficients (which is unstable)
>  > only to go back and compute the polynomial values at the same points
>  > you fit, it should be possible to set things up so the numerical
>  > instabilities are not really a problem - if some linear combination of
>  > polynomials gives nearly zero values, then you might as well just take
>  > it to be zero. If you want to look into this, you might try
>  > implementing your own polynomial fitting using the scipy.linalg.svd,
>  > which will give you control over how you deal with this situation.
>
> Thanks for this tip, again.

Ah, I should have read the docstring on polyfit. In fact it works in
just this way, so you can be reasonably confident that it will do the
right thing. If you want to suppress the error you're seeing, try
using "full=True" in your call to polyfit. This will return a number
of values you don't care about, but will suppress the warning (because
"rank" and "rcond" contain that information). It also contains the
advice that subtracting the mean of your data before starting is a
good idea.

>  As for splines, I've not tried that. I asked people before about splines
>  and they told me that they are even more unstable than polynomials,
>  however! so I didn't even try. Again, however, thank you to telling me that.

I think they must be talking about splines without smoothing. The most
usual kind of spline you see is forced to go through every data point,
which is clearly not going to work for noisy data. But with some
cleverness, implemented in scipy's splrep, you can find splines that
are a least-squares fit in a particular sense (something like the
straightest curve passing within one sigma of your data). In my
experience they are extremely well-behaved.

It sounds like the answer to your original question should have been
"go right ahead, you're fine". For what you're doing, polyfit should
be quite robust, and the warning should not be a problem.

Good luck,
Anne