[SciPy-User] How to estimate error in polynomial coefficients from scipy.polyfit?

[David Goldsmith]

On Thu, Mar 25, 2010 at 3:40 PM, Jeremy Conlin <jlconlin at gmail.com> wrote:

> Yikes!  This sounds like it may be more trouble than it's worth.  I
> have a few sets of statistical data that each need to have curves fit
> to them.


That's an awfully generic need - it may be obvious from examination of the
data that a line is inappropriate, but besides polynomials there are many
other non-linear models (which can be linearly fit to data by means of data
transformation) which possess fewer parameters (and thus are simpler from a
parameter analysis perspective).  So, the question is: why are you fitting
to polynomials?  If it's just to get a good fit to the data, you might be
getting "more fit" than your data warrants (and even if that isn't a
problem, you probably want to use a polynomial form different from "standard
form," e.g., Lagrange interpolators).  Are you sure something like an
exponential growth/decay or power law model (both of which are "more
natural," linearizable, two-parameter models) wouldn't be more appropriate -
it would almost certainly be simpler to analyze (and perhaps easier to
justify to a referee).

On this note, perhaps some of our experts might care to comment: what
"physics" (in a generalized sense) gives rise to a polynomial dependency of
degree higher than two?  The only generic thing I can think of is something
where third or higher order derivatives proportional to the independent
variable are important, and those are pretty uncommon.

DG


> I can see what the parameters are and they are similar
> between each set.  I want to know if the coefficients are within a few
> standard deviations of each other.  That's what I was hoping to get
> at, but it seems to be more trouble than I'm willing to accept.
> Unless anyone has a simpler solution.
>
> Thanks,
> Jeremy
> >
> > Anne
> >
> >> residuals, rank, singular_values, rcond : present only if `full` = True
> >>        Residuals of the least-squares fit, the effective rank of the
> scaled
> >>        Vandermonde coefficient matrix, its singular values, and the
> specified
> >>        value of `rcond`. For more details, see `linalg.lstsq`.
> >>
> >> I don't think any of these things are "design matrix" as you have
> >> indicated I need.  The documentation for linalg.lstsq does not say
> >> what rcond is unfortunately.   Any ideas?
> >>
> >> Jeremy
> >> _______________________________________________
> >> SciPy-User mailing list
> >> SciPy-User at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-user
> >>
> > _______________________________________________
> > SciPy-User mailing list
> > SciPy-User at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100325/77111824/attachment.html>