[AstroPy] Info LevMarLSQFitter
Zé Vinícius
jvmirca at gmail.com
Wed May 13 23:06:27 EDT 2020
Hi Peter,
If the model is linear in its parameters, then it’s a linear least-squares
> problem and
> you can find the best estimator directly (e.g., via matrix inversion or
> orthogonal
> decomposition), without needing something like Levenberg-Marquardt.
>
> https://en.wikipedia.org/wiki/Linear_least_squares
>
Exactly, for which LM will find the unique solution.
The fact that the galaxy-fitting problem *can* be (but is not guaranteed to
> be)
> non-convex does not mean gradient-descent methods like L-M are *useless*;
> it depends on what the actual fit-statistic (e.g., chi^2) landscape is
> like.
>
I'd say it's better to consider optimization problems to be non-convex
until proven otherwise.
And from Diana's experiment of getting different solutions for different
initial points, I'd consider
that a good indicator of non-convexity.
The primary justification for something like L-M is that it is *fast*. My
> experience
> with fitting galaxy images is that it is about an order of magnitude
> faster than
> the simplex and about *two* orders of magnitude faster than Differential
> Evolution, which is considered relatively fast for a genetic-algorithms
> approach.
> If the L-M fit converges in less than a second, then you don’t lose much
> with a
> slower but more robust algorithm. But if the L-M fit takes an hour to
> converge,
> then waiting ten or a hundred hours for convergence with another algorithm
> may be a bit of a problem. (Or if you want to fit several tens of
> thousands of
> different images…)
>
It seems to me that the key here is to have a reasonable initial guess for
the parameters of the Sersic2D model.
Diana:
I have figured out that if I change the initial parameters in ini_mod
> (i.e., ini_flux, ini_r_eff, etc), I counterintuitively obtain always a
> different result although the galaxy I am fitting is the same. How does the
> fit depend on those initial parameters?
>
Your problem is likely to be nonconvex, so it's not so counter-intuitive
that you get different results for different initial points: it's just
the nature of LM, it will converge to the nearest local minima.
Best,
Zé
On Thu, May 14, 2020 at 4:11 AM Peter Erwin <peter.erwin at gmail.com> wrote:
> Hi Zé,
>
> If the model is linear in its parameters, then it’s a linear least-squares
> problem and
> you can find the best estimator directly (e.g., via matrix inversion or
> orthogonal
> decomposition), without needing something like Levenberg-Marquardt.
>
> https://en.wikipedia.org/wiki/Linear_least_squares
>
>
> The fact that the galaxy-fitting problem *can* be (but is not guaranteed
> to be)
> non-convex does not mean gradient-descent methods like L-M are *useless*;
> it depends on what the actual fit-statistic (e.g., chi^2) landscape is
> like.
>
> The primary justification for something like L-M is that it is *fast*. My
> experience
> with fitting galaxy images is that it is about an order of magnitude
> faster than
> the simplex and about *two* orders of magnitude faster than Differential
> Evolution, which is considered relatively fast for a genetic-algorithms
> approach.
> If the L-M fit converges in less than a second, then you don’t lose much
> with a
> slower but more robust algorithm. But if the L-M fit takes an hour to
> converge,
> then waiting ten or a hundred hours for convergence with another algorithm
> may be a bit of a problem. (Or if you want to fit several tens of
> thousands of
> different images…)
>
>
> cheers,
>
> Peter
>
>
> > On May 13, 2020, at 6:59 PM, Zé Vinícius <jvmirca at gmail.com> wrote:
> >
> > Hi Peter,
> >
> > As an aside, the fact that the Sersic2D model is nonlinear in its
> parameters is formally
> > irrelevant, since Levernberg-Marquardt is by design a *non-linear
> least-squares* algorithm.
> > (Levenberg’s original 1944 paper describing the algorithm was titled "A
> Method for the Solution
> > of Certain Non-Linear Problems in Least Squares”.)
> >
> > The thing is that if the model were linear in its parameters, then the
> problem would be convex, and LM would find the global unique optimum
> > no matter the initial point. But if that were the case, I'd advise other
> methods to get that solution.
> >
> > Best,
> > Zé
> >
> > On Wed, May 13, 2020 at 10:42 PM Paul Barrett <pebarrett at gmail.com>
> wrote:
> > Personally, I would suggest ditching the Levenberg-Marquardt and Simplex
> algorithms and using an interior point method, which is more versatile and
> robust in finding the global minimum. Interior point methods more easily
> handle constraints and provide better convergence information. cvxopt is a
> nice Python package for second order problems, which appears to be your
> problem. However, if your problem is nonconvex, i.e., has local minima,
> then you might want to consider using a branch-and-bound algorithm, which
> will find the global minimum, although the algorithm is not as efficient.
> The algorithm also works with integer parameters, which are inherently
> nonconvex.
> >
> > Cheers,
> > Paul
> >
> >
> >
> > On Tue, May 12, 2020 at 8:36 AM Peter Erwin <peter.erwin at gmail.com>
> wrote:
> > Hi Zé,
> >
> > As an aside, the fact that the Sersic2D model is nonlinear in its
> parameters is formally
> > irrelevant, since Levernberg-Marquardt is by design a *non-linear
> least-squares* algorithm.
> > (Levenberg’s original 1944 paper describing the algorithm was titled "A
> Method for the Solution
> > of Certain Non-Linear Problems in Least Squares”.)
> >
> > In practice, I find that Levenberg-Marquardt *usually* finds the global
> minimum
> > in galaxy image-fitting problems (e.g., I get the same solution when
> running Imfit
> > with Levenberg-Marquardt or with the Differential Evolution algorithm,
> which
> > doesn’t use initial values), though there *are* cases where L-M does get
> trapped in
> > local minima. (The fact that Diana says she “always” gets a different
> result for
> > different starting parameters maybe suggests something else is the
> problem.)
> >
> >
> > Diana — have you tried using an alternate fitting algorithm, such as the
> SimplexLSQFitter
> > class? (Simplex fitters tend to be less likely to be trapped by local
> minima than
> > Levenberg-Marquardt, though they’re usually significantly slower.)
> >
> >
> > cheers,
> >
> > Peter [Erwin]
> >
> > > On May 12, 2020, at 11:17 AM, Zé Vinícius <jvmirca at gmail.com> wrote:
> > >
> > > Hi Diana,
> > >
> > > From my understanding, LevMarLSQFitter uses the least-squares function
> as the objective to be minimized using the Levenberg-Marquardt algorithm.
> > >
> > > Because Sersic2D is a non-linear model on the parameters to be fitted,
> the problem is most likely non-convex, which means that Levenberg-Marquardt
> only guarantees to find a local solution, which will depend on the initial
> value for those parameters.
> > >
> > > Best,
> > > Zé
> > >
> > > On Tue, May 12, 2020 at 4:53 PM Diana Scognamiglio <
> dianasco at astro.uni-bonn.de> wrote:
> > > Dear Astropy Staff,
> > >
> > > I am Diana Scognamiglio and I have some problems to deep understand
> how LevMarLSQFitter works.
> > >
> > > I am trying to fit a mock Sersic galaxy using the LevMarLSQFitter
> classe, coding these lines:
> > >
> > >
> > >
> > > ini_mod =
> > > EllipSersic2D(flux=ini_flux, r_eff=ini_r_eff, n=ini_n,
> x_0=stampsize/2.0, y_0=stampsize/2.0, g1=0.0, g2=0.0)
> > >
> > > fitter = fitting.LevMarLSQFitter()
> > >
> > > fit_mod = fitter(ini_mod, x_array, y_array, stamp, weights=weights,
> maxiter = 1000, acc=1.0e-7, epsilon=1.0e-6,
> > > estimate_jacobian=False)
> > >
> > >
> > > I have figured out that if I change the initial parameters in ini_mod
> (i.e., ini_flux, ini_r_eff, etc), I counterintuitively obtain always a
> different result although the galaxy I am fitting is the same. How does the
> fit depend on those initial parameters?
> > >
> > > Please, can you help me with this issue?
> > >
> > > Thank you in advance.
> > >
> > > I wait for hearing from you.
> > >
> > > Best regards,
> > >
> > > Diana Scognamiglio
> > >
> > >
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at python.org
> > > https://mail.python.org/mailman/listinfo/astropy
> > >
> > >
> > > --
> > > Zé Vinícius
> > > https://mirca.github.io
> > > _______________________________________________
> > > AstroPy mailing list
> > > AstroPy at python.org
> > > https://mail.python.org/mailman/listinfo/astropy
> >
> > =============================================================
> > Peter Erwin Max-Planck-Insitute for Extraterrestrial
> > erwin at mpe.mpg.de Physics, Giessenbachstrasse
> > tel. +49 (0)176 2481 7713 85748 Garching, Germany
> > fax +49 (0)89 30000 3495 https://www.mpe.mpg.de/~erwin
> >
> >
> >
> >
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
> >
> >
> > --
> > Zé Vinícius
> > https://mirca.github.io
> > _______________________________________________
> > AstroPy mailing list
> > AstroPy at python.org
> > https://mail.python.org/mailman/listinfo/astropy
>
> =============================================================
> Peter Erwin Max-Planck-Insitute for Extraterrestrial
> erwin at mpe.mpg.de Physics, Giessenbachstrasse
> tel. +49 (0)176 2481 7713 85748 Garching, Germany
> fax +49 (0)89 30000 3495 https://www.mpe.mpg.de/~erwin
>
>
>
>
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
--
Zé Vinícius
https://mirca.github.io
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20200514/36eefa60/attachment-0001.html>
More information about the AstroPy
mailing list