[SciPy-dev] Proposal for more generic optimizers (posted before on scipy-user)

[Matthieu Brucher]

Hi again


This seems to be a different question?
> One question is the question of optimizer design:
> should it take as an argument an object that provides
> a certain mix of services, or should it take instead
> as arguments the functions proiding those services.
> I am not sure, just exploring it.



For a object point of view, I think that the "thing" the more capable of
knowing how to optimize itself is an object function. It knows better of the
context than several functions/objects each responsible for the value,
gradient, ...


I am used to optimizers that take a function,
> gradient procedure, and hessian procedure as arguments.
> I am just asking whether *requiring* these to be bundled
> is the right thing to do.



I find it far more convinient this way. Here, there is only gradient and
hessian, but one could imagine adding hessianp that returns
hessian*gradient, and if this method is available, it is automatically used.
In my research, I use partial gradient - only a part of the gradient, for
some parameters, is computed -, and if I had to pass it to the optimizer
each time, it woulb be very cumbersome - mainly because I use several
optimizers used sequentially -
It is written more shortly, every argument in the constructor can be used by
the methods, ...


This design would not mean that I cannot pass as arguments
> methods of some object.  (I think I am responding to your
> objection here.)


If you want to pass a method of another object to the optimizer, I suppose
you have a design problem. How a method of another object can know better
that a method from the object being optimized ? But then, you could do this
by assigning the gradient method to your custom object - but as I said, it
would be very very awkward and not adviced from a strictly computer science
point of view -.
I really would want to have a precise example as well :D


Note that requiring bundling imposes an interface
> requirement on the bundling object.  This is not true
> if I just provide the functions/methods as arguments.



Yes, that's true, but for functions, you still have to provide them, the
only thing you have to provide that is not in the functions solution is that
every gradient or hessian must have the same name. Not much, knowing that
you have a more concise module.


> Perhaps a more precise example of the usefullness is needed ?
>
> Perhaps so.



Enclosed is a simple example, only with a gradient, deriving the hessian was
not useful for the example.
So it is a class that allows the estimation the parameters (a, b) in y = ax
+ b, knowing y and x. The cost function is based on the German McClure
"distance", so it has an additional parameters - and it does not converge
efficiently -
If I had to pass the gradient as a parameters, how could I pass efficiently
y, x and the GM parameter to the function ? - it really is a question, BTW -
Here, I can't forget a parameter, they are all needed at the construction,
so less error prone, more robust as every call is made with the same
parameters and more modular as my cost function is really one block.

But then I know that it is not something we, as scientists, are accustomed
to, but as Python is object oriented, we could use this. It was not possible
with Matlab, and I suppose that this impacted a lot on how we want to use
optimizers, something new is always more difficult to apprehend and use.

Matthieu

P.S. : I did not change the code at the moment, but I'm thinking about the
most efficient way to do it :)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20070314/5cbf361a/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: LineEstimation.py
Type: text/x-python
Size: 1970 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20070314/5cbf361a/attachment.py>