[SciPy-User] Optimization problem using fmin_slsqp
Qing Yu
yuqingpsy at gmail.com
Thu Apr 10 21:32:56 EDT 2014
Thanks so much!
Best,
Qing
On Thu, Apr 10, 2014 at 12:31 PM, Yuxiang Wang <yw5aj at virginia.edu> wrote:
> Hi Qing,
>
> Sorry that I have not read the whole thread in my previous postings
> and not noticing that you have already got your full code... I rewrote
> the code and it works now.
>
>
> ####################################################################################
>
> # -*- coding: utf-8 -*-
>
> """
>
> Created on Thu Apr 10 12:23:59 2014
>
> temp.py
>
> @author: Yuxiang Wang
>
> """
>
>
> import numpy as np
>
> import matplotlib.pyplot as plt
>
> from scipy.optimize import minimize
>
>
> # Function definitions
>
> def f(x,a):
>
> return a[0]*np.exp((a[2]**2)*np.cos(x-a[1])-1)+a[3]
>
>
> def residual(a,x,y):
>
> return np.linalg.norm(y-f(x,a))**2
>
>
> # Assign values
>
> x=[0,np.pi/6,np.pi/3,np.pi/2,2*np.pi/3,5*np.pi/6]
>
> y=[0.17,0.36,0.61,0.38,0.17,0.16]
>
> a0=[1,np.pi/3,np.pi/2,0]
>
>
> # Fit!
>
> bounds=((None, None), (0, np.pi), (0, None), (None,None))
>
> res = minimize(residual, a0, args=(x, y), method='L-BFGS-B', bounds=bounds)
>
>
> # Plot
>
> fig, axs = plt.subplots()
>
> axs.plot(x, y, '.k')
>
> axs.plot(x, f(x, res.x), '-r')
>
>
> ####################################################################################
>
> And notice that you can safely change method='L-BFGS-B' to 'SLSQP' and
> it worked perfectly as well.
>
> Best,
> Shawn
>
> On Thu, Apr 10, 2014 at 11:22 AM, Qing Yu <yuqingpsy at gmail.com> wrote:
> > Hi Shawn,
> >
> > Thanks for your detailed explanation! I'm actually not sure how to check
> > whether residual() is spitting out nan; and also the fitting result
> (a~0.1,
> > mu~1, sigma~1.57, b~0.06) I got without bounds still fell in the bounds I
> > defined [(None,None),(0,pi),(0,None),(None,None)].
> >
> > On the other hand I've tried the 'minimize' function. I mentioned it in
> my
> > reply to Matt's email and here it is:
> >> Hi Matt,
> >>
> >> Thanks so much! I've actually tried the 'minimize' function in
> >> scipy.optimize. There are several constrained optimization methods in
> >> 'minimize' and among them 'SLSQP' seems to be equivalent to the method
> >> used
> >> in fmin_slsqp; I got similar bad fitting results using these two
> >> functions.
> >> Meanwhile, using the 'L-BFGS-B' method in 'minimize', I can get similar
> >> good
> >> fitting results as 'leastsq'. But I'm not sure how these two algorithms
> >> differ.
> >
> > So yes I've tried the 'L-BFGS-B' method and it is working well. Also,
> using
> > the 'lmfit' package as Matt suggested, I can get same good fitting
> results
> > as 'L-BFGS-B'. The 'SLSQP' method, however, provides similar bad fitting
> > results as 'fmin_slsqp'. I think that is because they are using the same
> > algorithm (Sequential Least SQuares Programming).
> >
> > Best,
> > Qing
> > '
> > ""
> >
> >
> > On Wed, Apr 9, 2014 at 9:16 PM, Yuxiang Wang <yw5aj at virginia.edu> wrote:
> >>
> >> Hi Qing,
> >>
> >> I was thinking that your function residual() may give nan and this
> >> would make the fitting uncontrollable.
> >>
> >> I think that this may be the case because in your previous emails you
> >> mentioned the error:
> >> Iteration limit exceeded (Exit mode 9)
> >> Current function value: nan
> >> Iterations: 101
> >> Function evaluations: 1606
> >> Gradient evaluations: 101
> >> The current function value is a "nan".
> >>
> >> It might simply be that in the bounds you defined, the algorithm does
> >> not converge due to the nan; if you get rid of the bounds, it will go
> >> elsewhere for sure but that does not give what you want.
> >>
> >> Could you check that to see whether residual() function is spitting out
> >> nan?
> >>
> >>
> >> PS: I heard that minimize() should always be used and fmin_slsqp etc.
> >> should be deprecated... I personally do prefer minimize because then I
> >> can easily try some other algorithms like L-BFGS-B (this and SLSQP are
> >> my favorite two!).
> >>
> >> -Shawn
> >>
> >> On Wed, Apr 9, 2014 at 11:24 AM, Qing Yu <yuqingpsy at gmail.com> wrote:
> >> > Hi Shawn,
> >> >
> >> > Yes I was actually using a 'try...except...' command in my script.
> It's
> >> > simply because I want an output when the fitting procedure fails.
> Here's
> >> > the
> >> > part of the script:
> >> >
> >> > try:
> >> > out, fx, its, imode, smode =
> >> >
> >> >
> fmin_slsqp(residual,a0,args=(x,y),bounds=[(None,None),(0,pi),(0,None),(None,None)],
> >> > full_output=True)
> >> > except:
> >> > out = [nan, nan, nan, nan]
> >> >
> >> > But I suppose I won't get the nan output unless the fitting fails?
> >> >
> >> > Thanks!
> >> >
> >> >
> >> > Best,
> >> > Qing
> >> >
> >> > On Wed, Apr 9, 2014 at 11:00 AM, Yuxiang Wang <yw5aj at virginia.edu>
> >> > wrote:
> >> >>
> >> >> Hi Qing,
> >> >>
> >> >> In your description I see a line:
> >> >>
> >> >> Current function value: nan
> >> >>
> >> >> Is your function outputting a np.nan for certain inputs? That might
> be
> >> >> bad for the algorithm to work. Could you check that?
> >> >>
> >> >> -Shawn
> >> >>
> >> >> On Mon, Apr 7, 2014 at 4:56 PM, Qing Yu <yuqingpsy at gmail.com> wrote:
> >> >> > Hi Matt,
> >> >> >
> >> >> > Thanks so much for your help! I've now changed the residual
> function
> >> >> > as
> >> >> > below:
> >> >> >
> >> >> > def residual(a,x,y):
> >> >> > return sum((y-f(x,a))**2)
> >> >> >
> >> >> > the script seems to be working after the change, but I have another
> >> >> > problem
> >> >> > now:
> >> >> >
> >> >> > because I've set the upper and lower bounds of the parameters in
> >> >> > fmin_slsqp
> >> >> > (and that is why I use fmin_slsqp instead of leastsq):
> >> >> > bounds=[(None,None),(0,pi),(0,None),(None,None)], if I keep the
> >> >> > bounds,
> >> >> > I
> >> >> > get the error:
> >> >> > Iteration limit exceeded (Exit mode 9)
> >> >> > Current function value: nan
> >> >> > Iterations: 101
> >> >> > Function evaluations: 1606
> >> >> > Gradient evaluations: 101
> >> >> > I tried changing iter to 10000, the error still existed.
> >> >> > If I delete the bound option, I can get the successful output.
> >> >> > However,
> >> >> > the
> >> >> > fitted values are quite different from what I get using lsqcurvefit
> >> >> > in
> >> >> > matlab. Moreover, if I plot the data, the model fit is indeed worse
> >> >> > using
> >> >> > fmin_slsqp. I'm not sure if the difference is caused by the missing
> >> >> > bounds.
> >> >> > But the outputs from fmin_slsqp did not go out of bounds anyway.
> Does
> >> >> > anyone
> >> >> > have any idea why I could not set the bounds in fmin_slsqp?
> >> >> >
> >> >> > Thanks so much for any help!
> >> >> >
> >> >> > Best,
> >> >> > Qing
> >> >> >
> >> >> >
> >> >> > On Mon, Apr 7, 2014 at 3:48 PM, Qing Yu <yuqingpsy at gmail.com>
> wrote:
> >> >> >>
> >> >> >> Dear SciPy experts,
> >> >> >>
> >> >> >> I'm using the fmin_slsqp function in scipy.optimize to fit my
> data,
> >> >> >> I
> >> >> >> first defined the function f and the residual of fitting, and then
> >> >> >> used
> >> >> >> actual data x and y to fit the curve. However, I encountered an
> >> >> >> error
> >> >> >> that I
> >> >> >> could not explain.
> >> >> >> Here's a short version of my script:
> >> >> >>
> >> >> >> def f(x,a):
> >> >> >> return a[0]*exp((a[2]**2)*cos(x-a[1])-1)+a[3]
> >> >> >> def residual(a,x,y):
> >> >> >> return y-f(x,a)
> >> >> >>
> >> >> >> x=[0,pi/6,pi/3,pi/2,2*pi/3,5*pi/6]
> >> >> >> y=[0.17,0.36,0.61,0.38,0.17,0.16]
> >> >> >> a0=[1,pi/3,pi/2,0]
> >> >> >> out, fx, its, imode, smode =
> >> >> >>
> >> >> >>
> >> >> >>
> fmin_slsqp(residual,a0,args=(x,y),bounds=[(None,None),(0,pi),(0,None),(None,None)],
> >> >> >> full_output=True)
> >> >> >>
> >> >> >> I ended up with the error: "failed in converting 8th argument `g'
> of
> >> >> >> _slsqp.slsqp to C/Fortran array." Does anyone have any idea of why
> >> >> >> this
> >> >> >> happened?
> >> >> >>
> >> >> >> Thanks so much in advance for any help!
> >> >> >>
> >> >> >> Best,
> >> >> >> Qing
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Graduate Student
> >> >> > Qing Yu
> >> >> > Department of Psychological and Brain Sciences
> >> >> > Dartmouth College
> >> >> > Hanover, New Hampshire 03755, US
> >> >> >
> >> >> > _______________________________________________
> >> >> > SciPy-User mailing list
> >> >> > SciPy-User at scipy.org
> >> >> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Yuxiang "Shawn" Wang
> >> >> Gerling Research Lab
> >> >> University of Virginia
> >> >> yw5aj at virginia.edu
> >> >> +1 (434) 284-0836
> >> >> https://sites.google.com/a/virginia.edu/yw5aj/
> >> >> _______________________________________________
> >> >> SciPy-User mailing list
> >> >> SciPy-User at scipy.org
> >> >> http://mail.scipy.org/mailman/listinfo/scipy-user
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Graduate Student
> >> > Qing Yu
> >> > Department of Psychological and Brain Sciences
> >> > Dartmouth College
> >> > Hanover, New Hampshire 03755, US
> >> >
> >> > _______________________________________________
> >> > SciPy-User mailing list
> >> > SciPy-User at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >> >
> >>
> >>
> >>
> >> --
> >> Yuxiang "Shawn" Wang
> >> Gerling Research Lab
> >> University of Virginia
> >> yw5aj at virginia.edu
> >> +1 (434) 284-0836
> >> https://sites.google.com/a/virginia.edu/yw5aj/
> >> _______________________________________________
> >> SciPy-User mailing list
> >> SciPy-User at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-user
> >
> >
> >
> >
> > --
> > Graduate Student
> > Qing Yu
> > Department of Psychological and Brain Sciences
> > Dartmouth College
> > Hanover, New Hampshire 03755, US
> >
> > _______________________________________________
> > SciPy-User mailing list
> > SciPy-User at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-user
> >
>
>
>
> --
> Yuxiang "Shawn" Wang
> Gerling Research Lab
> University of Virginia
> yw5aj at virginia.edu
> +1 (434) 284-0836
> https://sites.google.com/a/virginia.edu/yw5aj/
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140410/a0620b5e/attachment.html>
More information about the SciPy-User
mailing list