[SciPy-Dev] Levenberg-Marquardt Implementation

[Benny Malengier]

2014-02-25 19:25 GMT+01:00 Pauli Virtanen <pav at iki.fi>:

> 25.02.2014 17:45, Benny Malengier kirjoitti:
> > This is present already, See
> >
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.leastsq.html
> >
> > which wraps: http://www.math.utah.edu/software/minpack/minpack
> > /lmder.html
>
> The current leastsq implementation does not support sparse Jacobians
> or constraints.
>

Problems with sparse Jacobians could be nice improvement, have not
encountered such problems yet, my outputs being typically measurements
depending on all parameters.

As to constrainst, typically I use:
1. if contstrained on [c,inf], rescale the parameter to p=c+exp(q), with q
new parameter
2. if constrained on an interval, rescale the parameter to p=arctan(q) with
some offset and scaling for the interval size and start.

Results of this are typically good.. One just has to keep in mind that the
covariance is then on the transformed parameters. So doing on the optimized
variables a run to determine the covariance of the original parameters is a
good idea. Doing such transformation in your own code for full control is
probably best, but allowing transformations in the python layer is a nice
idea.

One can alternatively just return a continuous penalty which scales with
the distance from the required interval, going to error of 1e15 or so out
of the requried interval, and then 1e15 + dist_to_interval e15. Also then,
constraining will happen normally good.

As to LM, it's actually a simple algorithm. I used to have a C
implementation before using python on minpack. Adapting lmder.or a cython
implemetation seems not too hard to add sparse. The problem would be a good
sparse implementation in fortran, not the LM algorithm. No experience on
that. Doing LM in cython should be fast too.

Benny


> MINPACK does dense QR factorizations, and this approach doesn't work
> well for problems where the number of variables is too big. This was one
> of our the GSoC topic ideas [1] --- if you have suggestions on how to
> improve these, please speak up.
>
> [1] https://github.com/scipy/scipy/wiki/GSoC-project-ideas
>
> --
> Pauli Virtanen
>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20140225/6ba175e8/attachment.html>