[SciPy-User] max likelihood

[David Goldsmith]

On Mon, Jun 21, 2010 at 3:17 PM, Skipper Seabold <jsseabold at gmail.com>wrote:

> On Mon, Jun 21, 2010 at 5:55 PM, David Goldsmith
> <d.l.goldsmith at gmail.com> wrote:
> > On Mon, Jun 21, 2010 at 2:43 PM, Skipper Seabold <jsseabold at gmail.com>
> > wrote:
> >>
> >> On Mon, Jun 21, 2010 at 5:34 PM, David Goldsmith
> >> <d.l.goldsmith at gmail.com> wrote:
> >> > On Mon, Jun 21, 2010 at 2:17 PM, eneide.odissea
> >> > <eneide.odissea at gmail.com>
> >> > wrote:
> >> >>
> >> >> Hi All
> >> >> I had a look at the scipy.stats documentation and I was not able to
> >> >> find a
> >> >> function for
> >> >> maximum likelihood parameter estimation.
> >> >> Do you know whether is available in some other namespace/library of
> >> >> scipy?
> >> >> I found on the web few libraries ( this one is an
> >> >> example http://bmnh.org/~pf/p4.html <http://bmnh.org/%7Epf/p4.html> )
> having it,
> >> >> but I would prefer to start playing with what scipy already offers by
> >> >> default ( if any ).
> >> >> Kind Regards
> >> >> eo
> >> >
> >> > scipy.stats.distributions.rv_continuous.fit (I was just working on the
> >> > docstring for that; I don't believe my changes have been merged; I
> >> > believe
> >> > Travis recently updated its code...)
> >> >
> >>
> >> This is for fitting the parameters of a distribution via maximum
> >> likelihood given that the DGP is the underlying distribution.  I don't
> >> think it is intended for more complicated likelihood functions (where
> >> Nelder-Mead might fail).  And in any event it will only find the
> >> parameters of the distribution rather than the parameters of some
> >> underlying model, if this is what you're after.
> >>
> >> Skipper
> >
> > OK, but just for clarity in my own mind: are you saying that
> > rv_continuous.fit is _definitely_ inappropriate/inadequate for OP's needs
> > (i.e., am I _completely_ misunderstanding the relationship between the
> > function and OP's stated needs), or are you saying that the function
> _may_
> > not be general/robust enough for OP's stated needs?
>
> Well, I guess it depends on exactly what kind of likelihood function
> is being optimized.  That's why I asked.
>
> My experience with stats.distributions is all of about a week, so I
> could be wrong. But here it goes... rv_continuous is not intended to
> be used on its own but rather as the base class for any distribution.
> So if you believe that your data came from say an Gaussian
> distribution, then you could use norm.fit(data) (with other options as
> needed) to get back estimates of scale and location.  So
>
> In [31]: from scipy.stats import norm
>
> In [32]: import numpy as np
>
> In [33]: x = np.random.normal(loc=0,scale=1,size=1000)
>
> In [34]: norm.fit(x)
> Out[34]: (-0.043364692830314848, 1.0205901804210851)
>
> Which is close to our given location and scale.
>
> But if you had in mind some kind of data generating process for your
> model based on some other observed data and you were interested in the
> marginal effects of changes in the observed data on the outcome, then
> it would be cumbersome I think to use the fit in distributions. It may
> not be possible.   Also, as mentioned, fit only uses Nelder-Mead
> (optimize.fmin with the default parameters, which I've found to be
> inadequate for even fairly basic likelihood based models), so it may
> not be robust enough.  At the moment, I can't think of a way to fit a
> parameterized model as fit is written now.  Come to think of it though
> I don't think it would be much work to extend the fit method to work
> for something like a linear regression model.
>
> Skipper
>

OK, this is all as I thought (e.g., fit only "works" to get the MLE's from
data for a *presumed* distribution, but it is all-but-useless if the
distribution isn't (believed to be) "known" a priori); just wanted to be
sure I was reading you correctly. :-)  Thanks!

DG
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20100621/73a9c40d/attachment.html>