[SciPy-User] Confidence interval for bounded minimization

[Christopher Jordan-Squire]

On Wed, Feb 22, 2012 at 2:02 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Wed, Feb 22, 2012 at 8:48 PM,  <josef.pktd at gmail.com> wrote:
>> On Wed, Feb 22, 2012 at 3:26 PM, Greg Friedland
>> <greg.friedland at gmail.com> wrote:
>>> Hi,
>>>
>>> Is it possible to calculate asymptotic confidence intervals for any of
>>> the bounded minimization algorithms? As far as I can tell they don't
>>> return the Hessian; that's including the new 'minimize' function which
>>> seemed like it might.
>>
>> If the parameter ends up at the bounds, then the standard statistics
>> doesn't apply. The Hessian is based on a local quadratic
>> approximation, which doesn't work if part of the local neigborhood is
>> out of bounds.
>> There is some special statistics for this, but so far I have seen only
>> the description how GAUSS handles it.
>>
>> In statsmodels we use in some cases the bounds, or a transformation,
>> just to keep the optimizer in the required range, and we assume we get
>> an interior solution. In this case, it is possible to use the standard
>> calculations, the easiest is to use the local minimum that the
>> constraint or transformed optimizer found and use it as starting value
>> for an unconstrained optimization where we can get the Hessian (or
>> just calculate the Hessian based on the original objective function).
>
> Some optimizers compute the Hessian internally. In those cases, it
> would be nice to have a way to ask them to somehow return that value
> instead of throwing it away. I haven't used Matlab in a while, but I
> remember running into this as a standard feature at some point, and it
> was quite nice. Especially when working with a problem where each
> computation of the Hessian requires an hour or so of computing time.
>

Are you talking about analytic or finite-difference gradients and
hessians? I'd assumed that anything derived from finite difference
estimations wouldn't give particularly good confidence intervals, but
I've never needed them so I've never looked into it in detail.

-Chris


> -- Nathaniel
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user