[SciPy-User] Saving Function Values in Optimize

[Sturla Molden]

If function evaluations is used to approximate the derivatives, you might
want to skip those evaluations in the tally, as they don't show the
convergence path. Then we end up with something like this (not tested, but
should work unless I made a typo), where the result object gets an 'fevals'
attribute that stores the function evaluation path, excluding approximation
of derivatives:


def minimize(f, x0, jac=None, hessp=None, epsilon=1.E-12, **kwargs):

    fevals = list()
    approx_fprime = sp.optimize.approx_fprime

    def fun(xk, *args):
        v = f(xk,*args)
        fevals.append(v)
        return v

    def approx_jac(xk, *args):
        return approx_fprime(xk, f, epsilon, *args)

    if jac is None:
        jac = approx_jac

    def approx_hessp(x0, p, *args):
        f2 = jac(*((x0 + epsilon*p,) + args))
        f1 = jac(*((x0,) + args))
        return (f2 - f1) / epsilon

    if hessp = None:
        hessp = approx_hessp

    res = scipy.optimize.minimize(fun, x0, args=args, 
        jac=jac, hessp=hessp, **kwargs)
    
    res.fevals = fevals 
    return res


:-)


Regards,
Sturla

Kevin Bache <kevin.bache at gmail.com> wrote:
> Thanks, Sebastian.  Good idea.
> Kevin
> 
> On Thu, May 8, 2014 at 4:04 PM, Sebastian Berg
> <sebastian at sipsolutions.net>wrote:
> 
>> On Do, 2014-05-08 at 12:56 -0700, Kevin Bache wrote:
>>> Hi Everyone,
>>> 
>>> Quick question: is there a preferred way to save function values in
>>> optimize.minimize?  The callback function format only passes 'xk', the
>>> current parameters for the optimization problem.  For some problem
>>> types, it's computationally trivial to convert that to f(xk), but in
>>> many, that process is expensive.  I'd like to save all function
>>> evaluations as I progress through the optimization process, yielding
>>> an object with the information:
>>> 
>>> 
>>> (xk_1, f(xk_1)), (xk_2, f(xk_2)), ...  (xk_n, f(xk_n))
>>> 
>>> 
>>> Does anyone have advice on how to go about this without hacking the
>>> optimize code?
>>> 
>> The sniplet below isn't perfect, but I think you should be able to adept
>> it to your needs. Decorators can do this kind of magic pretty nicely
>> (though of course nothing stops you from just implementing it into the
>> function itself).
>> 
>> - Sebastian
>> 
>> 
>> def store_cost(func):
>>     """Decorator for cost functions, as long as the cost function is only
>>     called with the same arguments this works good. Defines func.x and
>>     func.cost.
>> 
>>     Example:
>>     ========
>> 
>>     from scipy.optimize import fmin
>> 
>>     @store_cost
>>     def func(x):
>>         return (x - 10)**2
>>     fmin(func, [0])
>> 
>>     print func.x
>>     print func.cost
>>     """
>>     x_list = []
>>     cost_list = []
>>     def new_func(x, *args):
>>         x_list.append(x.copy())
>>         e = func(x, *args)
>>         cost_list.append(e)
>>         return e
>>     new_func.x = x_list
>>     new_func.cost = cost_list
>>     return new_func
>> 
>> 
>>> 
>>> Thanks in advance!
>>> 
>>> 
>>> Best Regards,
>>> Kevin
>>> 
>>> _______________________________________________
>>> SciPy-User mailing list
>>> SciPy-User at scipy.org
>>> <a
>>> href="http://mail.scipy.org/mailman/listinfo/scipy-user">http://mail.scipy.org/mailman/listinfo/scipy-user</a>
>> 
>> 
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> <a
>> href="http://mail.scipy.org/mailman/listinfo/scipy-user">http://mail.scipy.org/mailman/listinfo/scipy-user</a>
>> 
>> 
> 
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> <a
> href="http://mail.scipy.org/mailman/listinfo/scipy-user">http://mail.scipy.org/mailman/listinfo/scipy-user</a>