[SciPy-Dev] improvements of optimize package

[Christoph Deil]

On Sep 16, 2011, at 11:12 PM, Christopher Jordan-Squire wrote:

> On Thu, Sep 15, 2011 at 2:02 PM, Denis Laxalde <denis.laxalde at mcgill.ca> wrote:
>> Hi,
>> 
>> As discussed recently on the -user list [1], I've started thinking
>> about possible improvements of the optimize package, in particular
>> concerning the consistency of functions signature, parameters/returns
>> names, etc. I've posted a proposal on the wiki of my gihub account [2].
>> In brief, the proposal is two-fold:
>> 
>>  - First, concerning the standardization of functions signature, I
>>   would propose in particular to gather solver settings in a
>>   dictionary (named `options`) and to generalized the use of the
>>   `infodict`, `ier`, `mesg` outputs which respectively correspond
>>   solver statistics, exit flag and information message. I've tried
>>   also to choose simple yet informative variables names.
>> 
>>  - Then, as discussed in the aforementioned thread, the implementation
>>   of unified interfaces (or wrappers) to several algorithms with
>>   similar purpose is proposed. There definition is basically taken
>>   from the classification of the optimize package documentation [3].
>> 
>> I've also try to list the impact of the proposed changes on the
>> existing functions as well. Also, having started working on the code, I
>> would say that most of this is feasible. Yet, many choices are somehow
>> arbitrary and are thus subject to discussions so: comments welcome!
>> 
>> --
>> Denis Laxalde
>> 
>>  1: http://mail.scipy.org/pipermail/scipy-user/2011-September/030444.html
>>  2: https://github.com/dlaxalde/scipy/wiki/Improvements-to-optimize-package
>>  3: http://docs.scipy.org/doc/scipy/reference/optimize.html
>> 
>> 
> 
> Looks well thought-out. A few comments:
> 
> Is there a reason to give two interfaces for constrained and
> unconstrained optimization instead of combining them somehow?
> Why not include leastsq with the unconstrained optimization methods
> for the unconstrained optimization interface?
> Why not put anneal in with the unconstrained optimization interface?
> You could include in the interface the possibility of returning a
> dictionary of solver specific outputs.
> 
> Bound constrained optimization should be associated with constrained
> optimization, not unconstrained. Also, I think there's a typo in the
> multivariate solvers: 'newton\_krylov' should be 'newton_krylov'.
> 
> -Chris JS
> 

Hi Denis,

I also think your proposal is great.
Here are my comments (mostly on new names), in the order of the document:

- Why not have all optimizers return only two things, x and infodict? The difference to your proposal would be that "ier" and "mesg" as well as solver-dependent extra stuff would all be in infodict instead of having variable-length return tuples, which make it harder to quickly switch optimizers. If it's just a dict I can simply pprint it and as long as I don't look at items that are not available for all solvers, I only have to change one word in the code (the optimizer name) to switch to any other optimizer.
- Can we call the unified minimizer simply "minimize" instead of "fminunc", which is kind of hard to pronounce?
- Why not call the (optional) returned function, Jacobian and Hessian "fun", "jac" and "hess". This is easier to remember than the f, g, h you propose, because it is the same names as for the inputs.
- constrained minimizers could be "con_minimize" instead of "fmincon". Also: you call the parameters "cons" (e.g. "eq_cons"), but the  minimizer "con". Pick one or the other.
- Scalar function minimizers could be "minimize1d" instead of "fmins", similar to "interp1d" and other 1d specific stuff in scipy.
- I would avoid renaming fmin to something else, because it is so much in use. You could simply choose a new name for the root finding wrapper, e.g. "root".

Thanks for making a nicer scipy.optimize!

Christoph