[SciPy-dev] Abstract vectors in optimization

[Charles R Harris]

On Thu, Jan 8, 2009 at 7:59 AM, Ben FrantzDale <benfrantzdale at gmail.com>wrote:

> On Tue, Jan 6, 2009 at 10:56 PM, Robert Kern <robert.kern at gmail.com>wrote:
>
>> 2009/1/6 Ben FrantzDale <benfrantzdale at gmail.com>:
>> > David, Robert, et al.
>> >
>> > I think you understand what I am talking about. This is just about
>> > optimization (although in general I would argue that functions should
>> take
>> > as general an interface as possible for the same reasons).
>>
>> I'm not sure I would push it that far. Premature generalization can be
>> just as bad as premature optimization. I find that a use case-based
>> approach is a better principle. Write the code for the problem in
>> front of you. Generalize when you have a use case for doing so.
>>
>
> I completely agree; I was just speculating :-)
>
>
>> Now, we do have a handful of use cases here, but I think we should
>> have some care before modifying the functions in place. Generalization
>> could cost us performance in the typical case.
>>
>> > Attached is all the code I changed to make fmin_cg work with a
>> > SimpleHilbertVector class.
>>
>> Hmm, can you show us a .diff of the changes that you made? I'd like to
>> see more clearly exactly what you changed.
>>
>
> See attached.
> Run this:
> $ patch Hilbert_cg_example_original.py Hilbert_cg_example.diff -o
> Hilbert_cg_example.py
> to get Hilbert_example.py. (I can't fit all the files in one email to this
> list.)
>
> I cleaned it up a bit and included all the functions I changed, all in one
> file to make a diff make sense. The diff lines are almost all just
> numpy.dot(u,v) -> inner_product(u,v) and vecnorm(v,norm) -> v.norm(norm).
>
>
>> > If people are receptive to this approach, I have a few next questions:
>> > 1. What is the right interface to provide? Is the interface I described
>> the
>> > right (Pythonic) way to do it, or is there a better interface? (If it
>> were
>> > Templated C++, I'd do it with overloaded free functions, but I think it
>> > needs to/should be done with class methods.)
>>
>> I still don't think this interface supports the manifold-optimization
>> use case. The inner product and norm implementations need more
>> information than just the vectors.
>>
>
> Good point. I've been focused on optimizing within a vector space not on a
> manifold. For nonlinear CG, I suppose the gradient needs to be a vector in
> the tangent space, which itself must be a Hilbert space, but the state
> "vector" can be in an affine space or on a manifold. What is the
> mathematically-correct operation for moving such a "vector" on its manifold?
>

Moving vectors on manifolds is a can of worms, leading to covariant
differentiation via connections, parallel transport, Christoffel symbols,
etc., etc.. The general case also probably needs a set of coordinate maps
for computation. I'm not sure we want to go there. It's easier to embed the
manifold in a higher dimensional space and use vectors and inner products,
moving the vectors in the embedding space and projecting onto the manifold.
Control systems do quaternions ( 3 sphere) that way, reducing the problem to
constrained optimization. I suppose much depends much on the problem at
hand...

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090108/a87fbbff/attachment.html>