[SciPy-dev] SLSQP Constrained Optimizer Status

[Rob Falck]

I'm currently implementing the Sequential Least Squares Quadratic
Programming (SLSQP) optimizer, by Dieter Kraft, for use in Scipy.
The Fortran code being wrapped with F2PY is here:
http://www.netlib.org/toms/733  (its use within Scipy has been cleared)

SLSQP provides for bounds on the independent variables, as well as equality
and inequality constraint functions, which is a capability that doesn't
exist in scipy.optimize.
Currently the code works, although the constraint normals are being
generated by approximation.  I'm working on a way to pass these in.  I think
the most elegant way will be a single function that returns the matrix of
constraint normals.

For a demonstration of what the code can do, here is an optimization of
f(x,y) = 2xy + 2x - x**2 - 2y**2
Example 14.2 in Chapra & Canale gives the maximum as x=2.0, y=1.0.

The unbounded optimization tests find this solution.  As expected, its
faster when derivatives are provided rather than approximated.

Unbounded optimization. Derivatives approximated.
Elapsed time: 1.45792961121 ms
Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4,
0, 'Optimization terminated successfully.']

Unbounded optimization.  Derivatives provided.
Elapsed time: 1.03211402893 ms
Results [[1.9999999515712266, 0.99999996181577444], -1.9999999999999984, 4,
0, 'Optimization terminated successfully.']

The following example uses an equality constraint to find the optimal when
x=y.

Bound optimization.  Derivatives approximated.
Elapsed time: 1.384973526 ms
Results [[0.99999996845920858, 0.99999996845920858], -0.99999999999999889,
4, 0, 'Optimization terminated successfully.']

I've tried to conform to the syntax used by the other optimizers in
scipy.optimize.  The function definition and doc string are below.

If anyone is interested in testing it out, let me know.

def fmin_slsqp( func, x0 , eqcons=[], f_eqcons=None, ieqcons=[],
f_ieqcons=None,
                bounds = [], fprime = None, fprime_cons=None,args = (),
                iter = 100, acc = 1.0E-6, iprint = 1, full_output = 0,
                epsilon = _epsilon ):
    """
    Minimize a function using Sequential Least SQuares Programming

    Description:
        Python interface function for the SLSQP Optimization subroutine
        originally implemented by Dieter Kraft.

    Inputs:
        func         - Objective function (in the form func(x, *args))
        x0           - Initial guess for the independent variable(s).
        eqcons       - A list of functions of length n such that
                       eqcons[j](x0,*args) == 0.0 in a successfully
optimized problem
        f_eqcons     - A function of the form f_eqcons(x, *args) that
returns an
                       array in which each element must equal 0.0 in a
                       successfully optimized problem.  If f_eqcons is
                       specified, eqcons is ignored.
        ieqcons      - A list of functions of length n such that
                       ieqcons[j](x0,*args) >= 0.0 in a successfully
optimized problem
        f_ieqcons    - A function of the form f_ieqcons(x0, *args) that
returns an
                       array in which each element must be greater or equal
to
                       0.0 in a successfully optimized problem.  If
f_ieqcons is
                       specified, ieqcons is
ignored.
        bounds       - A list of tuples specifying the lower and upper bound
for each
                       independent variable [ (xl0, xu0), (xl1, xu1), ...]
        fprime       - A function that evaluates the partial derivatives of
func
        fprime_cons  - A function of the form f(x, *args) that returns the
                       m by n array of constraint normals.  If not provided,
                       the normals will be approximated. Equality constraint
                       normals precede inequality constraint normals.
        args         - A sequence of additional arguments passed to func and
fprime
        iter         - The maximum number of iterations (int)
        acc          - Requested accuracy (float)
        iprint       - The verbosity of fmin_slsqp.
                       iprint <= 0 : Silent operation
                       iprint == 1 : Print summary upon completion (default)
                       iprint >= 2 : Print status of each iterate and
summary
        full_output  - If 0, return only the minimizer of func (default).
Otherwise,
                       output final objective function and summary
information.
        epsilon      - The step size for finite-difference derivative
estimates.

    Outputs: ( x, { fx, gnorm, its, imode, smode })
        x            - The final minimizer of func.
        fx           - The final value of the objective function.
        its          - The number of iterations.
        imode        - The exit mode from the optimizer, as an integer.
        smode        - A string describing the exit mode from the optimizer.


    Exit modes are defined as follows:
        -1 : Gradient evaluation required (g & a)
         0 : Optimization terminated successfully.
         1 : Function evaluation required (f & c)
         2 : Number of equality constraints larger than number of
independent variables
         3 : More than 3*n iterations in LSQ subproblem
         4 : Inequality constraints incompatible
         5 : Singular matrix E in LSQ subproblem
         6 : Singular matrix C in LSQ subproblem
         7 : Rank-deficient equality constraint subproblem HFTI
         8 : Positive directional derivative for linesearch
         9 : Iteration limit exceeded






-- 
- Rob Falck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20071211/5d78b286/attachment.html>