[SciPy-user] openopt: which NLP solver?

[Emanuele Olivetti]

First of all thanks a lot Dmitrey. Your advices are
very valuable. Other comments below, inline.

dmitrey wrote:
> ...
> as well as many other solvers, including even ALGENCAN. Maybe I 'll 
> provide native OO handling of the situation but, on the other hand, 
> usually it means something incorrect with your own funcs, for example it 
> has lots of local minima or incorrect gradients.
>   

There are fair chances that there are many local minima (at
least "some" are present for sure) and some troubles with
gradients too :)

>> - it is possible that the evaluation of my function and gradient suffers
>> some numerical instabilities
>>   
>>     
> you should investigate is it so or not. Using p.check.df=1 could be very 
> helpful (see openopt doc page, "auto check derivatives" chapter). If 
> your 1st derivatives really have instabilities (noise, non-smooth - for 
> example, using abs(...)) - then only ralg can handle the problem. On the 
> other hand, current OpenOpt ralg implementation is still far from 
> perfect (at least from ralg fortran version by our dept). However, ralg 
> is for medium-scale problems with nVars up to ~1000, not 10000 as you 
> have. It handles matrix b of shape(nVars,nVars) in memory, and requires 
> 4..5*nVars^2 multiplication operations each iter.
> If no problems with smooth and noise and 1st derivatives obtain, I would 
> recommend you scipy_lbfgsb, ALGENCAN, scipy_tnc, maybe scipy_slsqp; 
> lincher also can serve, but it's very primitive. Here's full list: 
> http://scipy.org/scipy/scikits/wiki/NLP
>
>   

Very interesting. I'll investigate as you suggest. I'll work on
the 1000 variable problem and ralg in a short time. After that
I'll try the other solvers you suggested.

>> If someone (dmitrey?) could help selecting most appropriate solver
>> in openopt it would be much appreciated.
>> In the meawhile I'll try 'ralg'.
>>
>>
>> Thanks in advance,
>>
>> Emanuele
>>
>> P.S.: I'm having some troubles building openopt in ubuntu gutsy.
>> "sudo python setup.py install" works but "python setup.py build"
>> does not, requiring a previously installed "scikits" package. How
>> can I install openopt in a custom path instead of /usr/local/lib/... ?
>>   
>>     
> Try using "python setup.py", it must ask you what to do (cho0se install) 
> and destination (choose your own)
>   

Right, I'm just trying now. Cool :)

>>
>> [0]: by the way fmin_cg does not handle constraints, at least in
>> standard scipy, which forced me to use abs() in many places. This
>> could be a source of instabilities when computing gradient.
>>   
>>     
> I guess you use very obsolete OpenOpt version.
> I had implemented (ling time ago) yielding error message
> "the solver scipy_cg cannot handle 'lb' constraints"
> (at least I get the one with trying to pass lb constraints to scipy_cg)
> Of course, since scipy.optimize.fmin_cg can't handle constraints, so 
> does OO scipy_cg.
>
>   

The 'fmin_cg' I'm using is the standard scipy's one (scipy-0.5.2, as
in ubuntu gutsy 7.10), not yours in openopt. I'll use yours now.

Thanks again!

Emanuele