[SciPy-dev] fix for ncx2 bug?

[josef.pktd at gmail.com]

On Wed, Jun 17, 2009 at 1:08 PM, Neal Becker<ndbecker2 at gmail.com> wrote:
> josef.pktd at gmail.com wrote:
>
>> On Wed, Jun 17, 2009 at 10:45 AM, Neal Becker<ndbecker2 at gmail.com> wrote:
>>> As reported earlier, there is a bug in ncx2 that causes a strange
>>> discontinuity.
>>>
>>> Is there a patch available?
>>>
>>
>> It's a ticket, but it could take some time until someone finds all the
>> imprecision for "outlier" cases in scipy.special.
>>
>> I'm not sure these functions are even designed to have a high
>> precision, because for distributions such as chisquare or ncx2, that
>> are primarily used for test statistics, it make only sense to report a
>> few digits.
>>
>> I would have recommended ncx2.veccdf for your case, but a quick check
>> showed that over a large range ncx2.veccdf and ncx2.cdf only agree up
>> to 1e-8. For ncx2.cdf close to one, there might also be a precision
>> loss.
>>
>> What's your use case that you need ncx2 at high precision?
>>
>> Josef
>
> I'm using this to compute what's often called "Receiver Operating
> Characteristic", which is a signal detection problem.  You may see over some
> range that probability of miss or false detection is quite low.  Probably
> you don't really care about accuracy here, but it's making the plots look
> bad.

If speed doesn't matter too much, you can still use veccdf, I don't
think it's more accurate but it is smooth, no discontinuities and it
also goes to 1 in the upper tail.

veccdf is a private method, that avoids the argument check, but as
long as you call it with valid arguments you get good solutions, and
it is vectorized (based on np.vectorize). The generic methods are
pretty heavily tested, so besides slower speed from the numerical
integration, I wouldn't expect any problems.

In contrast to central chisquare, I didn't see any formulas that would
allow a direct calculation of the ncx2.cdf as function of some other
special functions.

Josef