[Matrix-SIG] Nonlinear optimization routines anyone?

[David Ascher]

> The conjugate gradient algorithm is probably about twenty lines or less of
> matrix/vector statements in Python, assuming you have a preconditioner you
> can express that way. So just code it up in Python. It will be fast enough,
> all the hard work is in the dot products and matrix multiplies.

CG is fine for some cases, but not all.  What I miss in Python is the
choices offered by e.g. Matlab's Optimization Toolbox:

 - Unconstrained Optimization
 - Quasi-Newton 
 - Least Squares
 - Nonlinear Least Squares
   - Gauss-Newton
   - Levenberg Marquart
 - Constrained optimization
   - Sequential quadratic programming
 - Multiobjective optimization

[taken from the TOC].

I suspect that if I knew enough about optimization I could code up the
subset that I need, but I'm not sure I'd trust my own code to do this...

--david