[AstroPy] Info LevMarLSQFitter

Peter Erwin peter.erwin at gmail.com
Wed May 13 16:10:31 EDT 2020 

Hi Zé,

If the model is linear in its parameters, then it’s a linear least-squares problem and
you can find the best estimator directly (e.g., via matrix inversion or orthogonal
decomposition), without needing something like Levenberg-Marquardt.

https://en.wikipedia.org/wiki/Linear_least_squares


The fact that the galaxy-fitting problem *can* be (but is not guaranteed to be)
non-convex does not mean gradient-descent methods like L-M are *useless*;
it depends on what the actual fit-statistic (e.g., chi^2) landscape is like.

The primary justification for something like L-M is that it is *fast*. My experience
with fitting galaxy images is that it is about an order of magnitude faster than
the simplex and about *two* orders of magnitude faster than Differential
Evolution, which is considered relatively fast for a genetic-algorithms approach.
If the L-M fit converges in less than a second, then you don’t lose much with a
slower but more robust algorithm. But if the L-M fit takes an hour to converge,
then waiting ten or a hundred hours for convergence with another algorithm
may be a bit of a problem. (Or if you want to fit several tens of thousands of
different images…)


cheers,

Peter


> On May 13, 2020, at 6:59 PM, Zé Vinícius <jvmirca at gmail.com> wrote:
> 
> Hi Peter,
> 
> As an aside, the fact that the Sersic2D model is nonlinear in its parameters is formally
> irrelevant, since Levernberg-Marquardt is by design a *non-linear least-squares* algorithm.
> (Levenberg’s original 1944 paper describing the algorithm was titled "A Method for the Solution 
> of Certain Non-Linear Problems in Least Squares”.)
> 
> The thing is that if the model were linear in its parameters, then the problem would be convex, and LM would find the global unique optimum
> no matter the initial point. But if that were the case, I'd advise other methods to get that solution.
> 
> Best,
> Zé
> 
> On Wed, May 13, 2020 at 10:42 PM Paul Barrett <pebarrett at gmail.com> wrote:
> Personally, I would suggest ditching the Levenberg-Marquardt and Simplex algorithms and using an interior point method, which is more versatile and robust in finding the global minimum. Interior point methods more easily handle constraints and provide better convergence information. cvxopt is a nice Python package for second order problems, which appears to be your problem. However, if your problem is nonconvex, i.e., has local minima, then you might want to consider using a branch-and-bound algorithm, which will find the global minimum, although the algorithm is not as efficient. The algorithm also works with integer parameters, which are inherently nonconvex.
> 
> Cheers,
> Paul
> 
> 
> 
> On Tue, May 12, 2020 at 8:36 AM Peter Erwin <peter.erwin at gmail.com> wrote:
> Hi Zé,
> 
> As an aside, the fact that the Sersic2D model is nonlinear in its parameters is formally
> irrelevant, since Levernberg-Marquardt is by design a *non-linear least-squares* algorithm.
> (Levenberg’s original 1944 paper describing the algorithm was titled "A Method for the Solution 
> of Certain Non-Linear Problems in Least Squares”.)
> 
> In practice, I find that Levenberg-Marquardt *usually* finds the global minimum
> in galaxy image-fitting problems (e.g., I get the same solution when running Imfit
> with Levenberg-Marquardt or with the Differential Evolution algorithm, which
> doesn’t use initial values), though there *are* cases where L-M does get trapped in
> local minima. (The fact that Diana says she “always” gets a different result for 
> different starting parameters maybe suggests something else is the problem.)
> 
> 
> Diana — have you tried using an alternate fitting algorithm, such as the SimplexLSQFitter
> class? (Simplex fitters tend to be less likely to be trapped by local minima than
> Levenberg-Marquardt, though they’re usually significantly slower.)
> 
> 
> cheers,
> 
> Peter [Erwin]
> 
> > On May 12, 2020, at 11:17 AM, Zé Vinícius <jvmirca at gmail.com> wrote:
> > 
> > Hi Diana,
> > 
> > From my understanding, LevMarLSQFitter uses the least-squares function as the objective to be minimized using the Levenberg-Marquardt algorithm.
> > 
> > Because Sersic2D is a non-linear model on the parameters to be fitted, the problem is most likely non-convex, which means that Levenberg-Marquardt only guarantees to find a local solution, which will depend on the initial value for those parameters.
> > 
> > Best,
> > Zé
> > 
> > On Tue, May 12, 2020 at 4:53 PM Diana Scognamiglio <dianasco at astro.uni-bonn.de> wrote:
> > Dear Astropy Staff,
> > 
> > I am Diana Scognamiglio and I have some problems to deep understand how LevMarLSQFitter works.
> > 
> > I am trying to fit a mock Sersic galaxy using the LevMarLSQFitter classe, coding these lines:
> > 
> >  
> >  
> > ini_mod =
> > EllipSersic2D(flux=ini_flux, r_eff=ini_r_eff, n=ini_n, x_0=stampsize/2.0, y_0=stampsize/2.0, g1=0.0, g2=0.0)
> > 
> > fitter = fitting.LevMarLSQFitter()
> > 
> > fit_mod = fitter(ini_mod, x_array, y_array, stamp, weights=weights, maxiter = 1000, acc=1.0e-7, epsilon=1.0e-6,
> > estimate_jacobian=False)  
> > 
> >  
> > I have figured out that if I change the initial parameters in ini_mod (i.e., ini_flux, ini_r_eff, etc), I counterintuitively obtain always a different result although the galaxy I am fitting is the same. How does the fit depend on those initial parameters?
> > 
> > Please, can you help me with this issue?
> > 
> > Thank you in advance.
> > 
> > I wait for hearing from you.
> > 
> > Best regards,
> > 
> > Diana Scognamiglio
> > 
> >  
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
> > 
> > 
> > -- 
> > Zé Vinícius
> > https://mirca.github.io
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
> 
> =============================================================
> Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
> erwin at mpe.mpg.de              Physics, Giessenbachstrasse
> tel. +49 (0)176 2481 7713     85748 Garching, Germany
> fax  +49 (0)89 30000 3495     https://www.mpe.mpg.de/~erwin
> 
> 
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
> 
> 
> -- 
> Zé Vinícius
> https://mirca.github.io
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy

=============================================================
Peter Erwin                   Max-Planck-Insitute for Extraterrestrial 
erwin at mpe.mpg.de              Physics, Giessenbachstrasse
tel. +49 (0)176 2481 7713     85748 Garching, Germany
fax  +49 (0)89 30000 3495     https://www.mpe.mpg.de/~erwin

More information about the AstroPy mailing list