[SciPy-Dev] optimize: what should happen if objective functions return non-finite numbers?

[Andrew Nelson]

Consider the following example which raises an AssertionError:

import numpy as np
from scipy.optimize import minimize
def func1(x):
    return np.nan
x0 = np.array([1.3, 0.7, 0.8, 1.9, 1.2])
res = minimize(func1, x0, method='l-bfgs-b')
assert(res.success = False)

minimize simply returns the starting values of: res.x == x0. The reason I
came up with this example is that unsanitised datasets sometimes contain
nan or inf. Thus, if func1 was calculating chi2 and you were using minimize
then the entire fit would appear to succeed (res.success is True), but the
output would be garbage. Ok, so res.message is CONVERGENCE:
NORM_OF_PROJECTED_GRADIENT_<=_PGTOL, but it's not an clear indicator that
something went wrong.
A second example is:

import numpy as np
from scipy.optimize import curve_fit
def func2(x, a, b, c):
    return a * np.exp(-b * x) + c

def func3(x, a, b, c):
    return np.nan

xdata = np.linspace(0, 4, 50)
y = func2(xdata, 2.5, 1.3, 0.5)
ydata = y + 0.2 * np.random.normal(size=len(xdata))

popt, pcov = curve_fit(func3, xdata, ydata)
print(popt)

Whilst there is a warning (OptimizeWarning: Covariance of the parameters
could not be estimated) it's not a clear indicator that something has gone
wrong.
The behaviour one might expect in both examples could be to see a
ValueError raised if there arenp.nan values returned from the objective
function. I'm not totally sure of what to do if +/- np.inf is returned
(-inf would be a very good global minimum).


--
_____________________________________
Dr. Andrew Nelson


_____________________________________
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20160615/1ec5735c/attachment.html>