[SciPy-dev] optimizers module

[dmitrey]

hi all,
I'm trying to connect a new optimizer (line-search via cubic 
interpolation) to Matthieu's code.
Here are some things that should be discussed, I guess in wide circle.
I have already committed
./solvers/optimizers/line_search/qubic_interpolation.py
tests/test_qubic_interpolation.py

the problems:
1. I have implemented stop tolerance x as self.minStepSize.
However, isn't it more correct to observe |x_prev - x_new| according to 
given by user xtol, than to observe |alpha_prev - alpha_new| according 
to xtol? If the routine is called from a multi-dimensional NL problem 
with known xtol, provided by user, I think it's more convenient and more 
correct to observe |x_prev - x_new| instead of |alpha_prev - alpha_new| 
as stop criterion.

2. (this is primarily for Matthieu): where should be gradtol taken from? 
It's main stop criterion, according to alg.
Currently I just set it to 1e-6.

3. Don't you think that maxIter and/or maxFunEvals rule should be added? 
(I ask because I didn't see that ones in Matthieu's 
quadratic_interpolation solver). It will make algorithms more stable to 
CPU-hanging errors because of our errors and/or special funcs encountered.
I had implemented that ones but since Matthieu didn't have them in 
quadratic_interpolation, I just comment out the stop criteria (all I can 
do is to set my defaults like 400 or 1000 (as well as gradtol 1e-6), but 
since Matthieu "state" variable (afaik) not have those ones - I can't 
take them as parameters).
So should they exist or not?

regards, D.