[SciPy-Dev] Levenberg-Marquardt Implementation

[Sturla Molden]

<josef.pktd at gmail.com> wrote:

> lmfit allows fancier parameter definitions, imposes constraints
> through reparameterization and produces more results statistics. The
> optimizer that it uses is just Levenberg-Marquardt, although IIRC it
> also wraps other optimizers.

Yes, and since we were talking about Levenberg-Marquardt, trust-region
optimizers are among the ones I miss...

Among the one-dimensional minimizers Newton-Raphson is a basic method which
is missing, though we have it for root-finding. While it is possible to
search for the root of the first derivative, but it would be better to have
a convergence check for function minimization instead.

Multidimensional Newton-Raphson is also missing. Though multidimensional
root-finding is dubious business, it is sometimes useful as a minimizer
when the Hessian is known. 

We could probably also use some form of the EM algorithm in scipy.optimize.

(Yes I know what to do with it, just post a PR...) 

Sturla