[SciPy-user] optimize.leastsq

[David Cournapeau]

On Fri, Jan 23, 2009 at 12:47 AM,  <josef.pktd at gmail.com> wrote:
> On Thu, Jan 22, 2009 at 10:03 AM, David Trethewey <dlrt2 at ast.cam.ac.uk> wrote:
>> Managed to get this working with my data. Being able to do a 2-d fit
>> with both metallicity and velocity information used is certainly
>> interesting, although it doesn't seem to be too good at detecting
>> subpopulations within my stellar stream which is what I'm trying to do.
>>
>
> >From my experience with hidden Markov models (estimated with ML not
> EM),

Well, EM, at least as implemented in the learn scikits, is
fundamentally a likelihood based method (EM is built such as its
objective function "force" a likelihood increase when it is itself
increased).

> I know that good starting values for the location parameters are
> necessary to get reliable results. I think, that the global properties
> of the likelihood function are not very "nice".

Indeed, for mixture with > 1 component, the likelihood function is not
concave anymore. EM can only find a local maximum of the likelihood.
The starting values are indeed important, there are various heuristics
which can help, but none of them are implemented in the toolbox.
Generally, one trick is to make sure the initial means are as far as
possible from each other - this is not always easy to do
automatically, although in that particular case, if the data are 2 d
with 2 components, this can be done by hand quite easily.

David