[Numpy-discussion] Does np.std() make two passes through the data?

[josef.pktd at gmail.com]

On Mon, Nov 22, 2010 at 12:28 PM, Keith Goodman <kwgoodman at gmail.com> wrote:
> On Mon, Nov 22, 2010 at 9:13 AM,  <josef.pktd at gmail.com> wrote:
>
>> Two pass would provide precision that we would expect in numpy, but I
>> don't know if anyone ever tested the NIST problems for basic
>> statistics.
>
> Here are the results for their most difficult dataset. But I guess
> running one test doesn't mean anything.
>
> http://www.itl.nist.gov/div898/strd/univ/addinfo/numacc4.html
>
>>> np.absolute(a.std(ddof=1) - 0.1)
>   5.5884095961911129e-10
>>> np.absolute(nanstd_online(a, ddof=1) - 0.1)
>   5.5890501948763216e-10
>>> np.absolute(nanstd_simple(a, ddof=1) - 0.1)
>   nan  # Ha!
>>> np.absolute(nanstd_twopass(a, ddof=1) - 0.1)
>   5.5879308125117433e-10

Thanks, e-10 is better than I expected for a tough test, but confirms
that I don't trust any statistics by more than 6 to 10 decimals or
digits.

Josef



> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>