[SciPy-User] non-linear function collection ?

[josef.pktd at gmail.com]

On Sat, Mar 8, 2014 at 6:15 PM, Matt Newville <newville at cars.uchicago.edu>wrote:

> Hi Josef,
>
> On Sat, Mar 8, 2014 at 12:30 PM,  <josef.pktd at gmail.com> wrote:
> > I'm trying out again some examples for nonlinear estimation.
> >
> > statsmodels still doesn't have nonlinear leastsquares, but now I'm trying
> > out robust estimation, e.g.
> > https://groups.google.com/d/msg/pystatsmodels/DPibQlUJmRA/arRlamlNivcJ
> >
> > Question since other packages have much more support for this:
> >
> > Is there a collection of frequently used non-linear functions?
> > including analytical derivatives, and self-starting, automatically
> created
> > starting values for numerical optimization?
>
> We're trying to do something along these lines with lmfit-py, with the
> goal of providing easy-to-use "simple fitting models".   We haven't
> really settled yet on the best final design (and there is some
> duplicated efforts), but we'd be open for suggestions. Currently, you
> might find the code at
>   https://github.com/lmfit/lmfit-py/blob/master/lmfit/model.py
>
> and
>   https://github.com/lmfit/lmfit-py/blob/master/lmfit/models1d.py
>
> useful.
>
> An attempt at 'canonical definitions' of such simple functions
> (inevitably incomplete) is at
>
> https://github.com/lmfit/lmfit-py/blob/rationalize_models/lmfit/utilfuncs.py
>  (note: non-master branch)
>
> The code in models1d.py above does have automated initial guesses for
> parameter values.    We haven't (yet?) added analytic derivatives, but
> that could be done.  In the lmfit approach, analytic derivatives are
> made extra challenging since each Parameter may be fixed, bounded, or
> constrained as an expression of other Parameters.
>

Thanks Matt, that's what I was looking for.

Sorry for the late response, I'm getting too side tracked these days.
I'm still trying to figure out how to get non-linear models into all or
many of the estimation models that statsmodels has or should get, and what
the statistics of it are. My main interest right now are robust estimators.


visiting some older packages again:

http://astropy.readthedocs.org/en/latest/modeling/#module-astropy.modeling.functional_models
has also derivatives
http://astropy.readthedocs.org/en/latest/api/astropy.modeling.functional_models.Beta1D.html#astropy.modeling.functional_models.Beta1D.fit_deriv

zunzun/pyeq2 has the largest collection of functions that I know, but it's
a bit hard to read because it supports the website and code generation.
for example
https://code.google.com/p/pyeq2/source/browse/trunk#trunk%2FModels_2D


I' was just looking at non-linear models again, and my preferred solution
for statsmodels would be to free-ride on some of these functions
collections by adding a wrapper for compatibility. I don't know much about
which kind of non-linear functions users are using.
I would be more interested in modelling when some of the parameters depend
on explanatory variables, for example the maximum and the speed of growth
in the sigmoid as function of a linear combination of explanatory
variables.

for example:
statsmodels has a collection of monotonic one parameter
functions/transformations that are used as link functions in generalized
linear models. y = f(eta)  where eta = x dot beta
https://github.com/statsmodels/statsmodels/blob/master/statsmodels/genmod/families/links.py
they define function, inverse function plus both derivatives.

for derivatives: I was using in my examples explicitly coded chain rules,
and using numerical derivatives for those pieces for which I didn't want to
figure out or hardcode the derivatives.
I didn't look at parameter transformation for bounds yet, but I guess it
can also be done by chaining, although that can get tricky

If my quick browsing is correct, you have the derivatives already
https://github.com/lmfit/lmfit-py/blob/35502f74e12a1f4155c2311d4530c38c7cc04293/lmfit/parameter.py#L156
I guess you use the derivatives of the bounding transformation in the
covariance calculation.


Josef



>
> Hope that helps,
>
> --Matt
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20140314/2f604f4f/attachment.html>