[SciPy-User] Sometimes fmin_l_bfgs_b tests NaN parameters andthen fails to converge

[Yury V. Zaytsev]

Hi!

On Tue, 2011-01-04 at 11:22 +0200, Dmitrey wrote:

> ralg (as well as gsubg) is adjusted very well to handle problems with
> restricted domains.

gsubg is listed as "unconstrained" at [2], while [3] says that
constraints handling code is immature and will most likely fail.

I still have no excuse for not trying r-alg out, which I will hopefully
do at some point, after exploring the performance of the nlopt solvers.

> For your problem (nonlinear local minimization) for a solver
> efficiency matter numbers of: variables, box constraints, linear
> eq/ineq constraints, nonlinear eq/ineq constraints. Another one very
> essential issue is having some gradients of active constraints forming
> linear system close to singular (then lots of NLP solvers will fail to
> solve it). Are you capable of taking into account all these
> parameters? I guess it's easy to change solver from the "stuff"
> instead and try what is better for the nlp involved. 

I am certainly not, because this is pretty much my first encounter with
optimization problems. Last week I didn't even have a clue on what
classes of problems there are, and which algorithms are mostly used to
solve these problems.

That's exactly why I would like to see descriptions along the lines of
[1], because random trials of the algorithms that are bundled with SciPy
(bfgs, tnc, ds, etc.) were largely unsuccessful.

At some point, BFGS was converging fast and nice, until I found out that
for different starting points it converges to completely different
solutions. Then I tried DS which proved to be much more robust, but
extremely slow and was all the time getting stuck when the simplex had
to get through a "needle's eye" in the parameter space.

So I assume that the procedure for choosing the right method should be
more of an educated guess, than random trials. Maybe I should re-post
the message more specific to my problem to this list.

> Moreover, would I provide some comparison info in the way you would
> like, after new release of any of the openopt-connected solver I would
> have to perform the comparison over and over again. Thus I don't see
> any reason to perform it.

Writing reasonably good documentation is not fun. Also, it takes a lot
of time that you can possibly spend on something else, which you might
find more interesting or useful. On the other hand, it makes users
happy, especially those that are unfamiliar with the domain.

What you find more important is entirely up to you, and please note that
I don't blame you for the decisions that you make. However, I find that
statements such as "I see no reason to perform it" are provocative :-)

If you want to put yourself in your users' shoes, just compare [1] and
[2]. Do you honestly think that there is no reason for coming up with a
similar list (short description of the algorithm, paper references and
personal recommendations for initial orientation) for OpenOpt? I think
it would certainly benefit the project if only you had time to do it.

[1]: http://ab-initio.mit.edu/wiki/index.php/NLopt_Algorithms
[2]: http://openopt.org/NLP
[3]: http://openopt.org/gsubg
 
-- 
Sincerely yours,
Yury V. Zaytsev