[SciPy-User] Optimization with smoothing

[josef.pktd at gmail.com]

On Sat, May 8, 2010 at 12:14 PM, Skipper Seabold <jsseabold at gmail.com> wrote:
> On Tue, May 4, 2010 at 9:34 AM, Angus McMorland <amcmorl at gmail.com> wrote:
>> Hi all,
>> I need to do some optimization where one of the parameters is a
>> spline-smoothed 1-d sequence, with, say, 10 values. What's the best way to
>> go about this using scipy (or any other numpy-compatible Python package)? I
>> could imagine using one of the scipy.optimize routines and then smoothing
>> the relevant parameters within the optimization loop, but it would be best
>> if the next iteration's of parameters were chosen from the previous
>> iteration's _smoothed_ parameters rather than their 'non-smooth'
>> predecessors, as it seems like this would keep the optimization better
>> behaved. Is this possible?
>
> I would think you could modify the callback function in the source of
> your chosen optimization routine from
>
> callback(xk)
>
> to
>
> xk = callback(xk)
>
> Though you would probably want to recompute the gradient and Hessian
> at the new smoothed parameters.  Sorry, I don't have a better answer,
> but I've often wondered the same thing and I'm hoping someone might
> know better than I.

I was wondering more whether you really have a well defined optimization
problem if you don't really use the parameters. Does the argmin really
end up at the smoothed values, or at some parameterization of the spline?

I would attempt to put the smoothness restriction in a constraint or
rewrite the problem in terms of some lower dimensional "hyper parameters".

Doing it directly might require adjustments to the optimization algorithm,
e.g. how a new point is found, so that it ends up hardcoding the
smoothness constraint into the optimization function.

My impression, and 2.5 cents,

Josef


>
> Skipper
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>