[SciPy-Dev] WIP: Class based Optimizers

[Andrew Nelson]

The #8414 PR on github (https://github.com/scipy/scipy/pull/8414)
introduces an `Optimize` class for `scipy.optimize`. The `Optimize` class
can be advanced like an iterator, or be advanced to convergence with the
`solve` method.
The PR is intended to provide a cleaner interface to the minimisation
process, with the user being able to interact with the solver during
iteration. This allows the variation of solver hyper-parameters, or other
user definable interactions. As such `callback`s become redundant, as you
can access all the optimizer information required.
Other historical parameters may become redundant in these objects, such as
`disp`. This is also a chance for a ground-zero approach to how people
interact with optimizers.

The PR is intended as a Request For Comment to gain feedback on
architecture of the solvers. Given that it's a key part of infrastructure
this may create a large amount of feedback, if I don't reply to it all bear
in mind that I don't have unlimited time resource.
In it's current state I've implemented `Optimizer`, `Function`, `LBFGS` and
`NelderMead` classes. `LBFGS` and `NelderMead` are being called by the
corresponding functions already in existence. The current test suite
passes, but no unit tests have been written for the functionality. I'd
prefer to add such a specific test suite once the overall architecture is
settled.

These optimisers were straightforward to translate. However, I don't know
how this approach would translate to other minimisers, or things like
`least_squares`. If a design could be agreed, then perhaps the best
approach would be to tackle the remaining optimizers on a case by case
basis once this PR is merged.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180214/2cd16ec8/attachment.html>