[SciPy-Dev] minimize(): tolerance unification

[Pauli Virtanen]

Hi,

20.05.2012 16:44, Matt Newville kirjoitti:
> If you're working on unifying the signatures of the various minimizer
> functions in scipy.optimize (Thanks!! Great!!), I have a small request
> to try to make the behavior of the user-supplied objective functions
> more uniform as well.

This work was mostly done by Dennis Laxalde, I've only done some minor
fiddling around here.

[clip]
> Currently, most solvers expect the objective function to return a
> scalar,  and raise a ValueError if the objective function returns an
> array.   OTOH, scipy.optimize.leastsq() requires the objective
> function to return an array.  As far as I can tell, the docs are
> actually silent on what the value returned from the objective function
> should be.

Good point. If this is not immediately clear after reading the
documentation, they should be improved.

[clip]
> Is it worth considering the following change?:  For all solvers
> expecting a scalar result, if the objective function returns an array,
>  convert to (array**2).sum().
> 
> Obviously, this implies a 'least squares' approach as the default,
> which is debatable but also probably defensible.    I believe this
> change would not break existing code (since the user can always return
> a scalar value), but it would allow more consistent objective
> functions, making it easier for users to try out different algorithms
> with the same objective function.

My first thought on this is that it would be mixing two separate things
--- nonlinear least squares, and optimization of scalar functions, and
I'm not so sure how desirable this is.

However, what we should have is a separate interface for least squares.
Most preferably, a thin wrapper to the solver routines themselves.
Currently, there is only one actual dedicated routine for this,
Levenberg-Marquardt, but it might make sense to do as lmfit does and
make it easy to use optimization solvers here (which would make it
easier for users to see how to do LSQ with constraints).

What to then do with the question of fancier higher-level interfaces to
fitting is not so clear to me --- curve_fit made it in, and two
different object oriented approaches have been proposed (your lmfit, and
the PR from Martin Teichmann). There are libraries such as openopt and
others which take a more structured approach, and I'm not clear on how
much such stuff should be in Scipy.

-- 
Pauli Virtanen