[SciPy-User] Why do SciPy stats return very small but very high precision numbers?

[Paul van Mulbregt]

Hi Vicki,
Probabilities are often computed using a formula  that just returns a number and that number might be very small. E.g. the two-sided Kolmogorov-Smirnov test for large samples uses a theta function from scipy.special which can return probabilities as small as 1e-308. No attempt is made to decide that a value is so small that it should be replaced with 0. 
On the other hand if the probability is computed as p=1-q, where q is the output of the formula, then the smallest non-zero value returned will be about 1e-16.  
Some of the test/distributions compute the CDF, some the SF and some can do both.
I think returning a very small non-zero probability value is the correct thing to do and then let the user decide what to do with it. It may well be that a very small probability is sending a signal that the assumptions made in deciding to use this particular test are not valid. Or it could just be saying that the sample is very far removed from the target distribution. Either way it's up to the user who has the sample data to decide.

Paul

> On Oct 15, 2019, at 3:11 PM, Vicki Brown <vlb at cfcl.com> wrote:
> 
> Sometimes, I see results like this:
> 
> LeveneResult(statistic=1233.9270599281986, pvalue=6.212186164025708e-269) 
> 
> Is it even realistic to claim a value that is this small? (6.212 e-269)?
> 
> Other times, a function just returns 0.0.
> 
> Can someone point me to an explanation of how and why the functions decide o return 0 ("fer sure") vs "almost but not exactly 0, but close enough")
> 
> -- Vicki
> 
> Vicki Brown
> cfcl.com/vlb
> 
> 
> 
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20191016/ef938666/attachment.html>