[SciPy-Dev] On the leastsq/curve_fit method

[josef.pktd at gmail.com]

On Tue, Sep 27, 2011 at 5:17 AM, Gianfranco Durin <g.durin at inrim.it> wrote:

>
> > where func can be both "_general_function" and
> > "_weighted_general_function", is not correct.
> >
> >
> > M = σ 2 I ok unit weights
> >
> > M = W^(-1) your case, W has the estimates of the error covariance
> >
> > M = σ 2 W^(-1) I think this is what curve_fit uses, and what is in
> > (my) textbooks defined as weighted least squares
> >
> > Do we use definition 2 or 3 by default? both are reasonable
> >
> > 3 is what I expected when I looked at curve_fit
> > 2 might be more useful for two stage estimation, but I didn't have
> > time to check the details
>
> Ehmm, no, curve_fit does not use def 3, as the errors would scale with W,
> but they don't. By the way, it does not have the correct units.
>

http://wwwasdoc.web.cern.ch/wwwasdoc/minuit/node31.html

'''
The minimization of [image: $ \chi^{2}_{}$] above is sometimes called *weighted
least  squares* in which case the inverse quantities 1/*e*2 are called the
weights. Clearly this is simply a different word for the same thing, but in
practice the use of these words sometimes means that the interpretation of *
e*2 as variances or squared errors is not straightforward. The word weight
often implies that only the relative weights are known (``point two is twice
as important as point one'') in which case there is apparently an unknown
overall normalization factor. Unfortunately the parameter errors coming out
of such a fit will be proportional to this factor, and the user must be
aware of this in the formulation of his problem.
'''
(I don't quite understand the last sentence.)

M = σ^2 W^(-1), where σ^2 is estimated by residual sum of squares from
weighted regression.

W only specifies relative errors, the assumption is that the covariance
matrix of the errors is *proportional* to W. The scaling is arbitrary. If
the scale of W changes, then the estimated residual sum of squares from
weighted regression will compensate for it. So, rescaling W has no effect on
the covariance of the parameter estimates.

I checked in Greene: Econometric Analysis, and briefly looked at the SAS
description. It looks like weighted least squares is always with automatic
scaling, W is defined only as relative weights.

All I seem to be able to find is weighted least squares with automatic
scaling (except for maybe some two-step estimators).


>
> Curve_fit calculates
>
> M = W \sigma^2 W^(-1) = \sigma^2
>

If I remember correctly (calculated from the transformed model) it should
be:

the cov of the parameter estimates is s^2 (X'WX)^(-1)
error estimates should be s^2 * W

where W = diag(1/curvefit_sigma**2)   unfortunate terminology for
curve_fit's sigma or intentional ? (as I mentioned in the other thread)

Josef


>
> so it gives exactly the same results of case 1, irrespective the W's. This
> is why the errors do not scale with W.
>
> Gianfranco
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20110927/4b103a5a/attachment.html>