[SciPy-Dev] ttest_1samp strange behaviour

[josef.pktd at gmail.com]

On Fri, May 20, 2011 at 5:11 PM, James Bergstra
<james.bergstra at gmail.com> wrote:
> I appreciate that I presented a corner case, but your example gives
> something even more bizarre: the two-tailed p-value jumps erratically almost
> all the way between 0 and 1 on adjacent calls!  Shouldn't it hold steady and
> approach 1?

(I might have to check, but I'm pretty sure)

Under the Null the p-values are uniform distributed.

try for power and you should see the monotonous decrease:

>>> for i in range(2,60,2): print(stats.ttest_1samp(1e-10+1e-10*np.random.randn(i), 0))

Josef

>
> On Fri, May 20, 2011 at 5:04 PM, <josef.pktd at gmail.com> wrote:
>>
>> On Fri, May 20, 2011 at 4:56 PM, James Bergstra
>> <james.bergstra at gmail.com> wrote:
>> > I was trying to make sure that ttest_1samp did what I thought and came
>> > across the following...
>> > In [41]: for i in range(10):
>> >    ....:     print scipy.stats.ttest_1samp([0]*i, 0)
>>
>> I don't know if this makes sense, variance is zero in your case
>>
>> try something like
>>
>> for i in range(2,1000,10):
>> print(stats.ttest_1samp(1e-10*np.random.randn(i), 0))
>>
>> Josef
>>
>> >    ....:
>> >    ....:
>> > (nan, nan)
>> > (1.0, nan)
>> > (1.0, 0.49999999999999956)
>> > (1.0, 0.42264973081037427)
>> > (1.0, 0.39100221895577053)
>> > (1.0, 0.37390096630005898)
>> > (1.0, 0.36321746764912255)
>> > (1.0, 0.35591768374958205)
>> > (1.0, 0.35061666282020748)
>> > (1.0, 0.34659350708733416)
>> > Am I interpreting this correctly that the null-hypothesis is decreasing
>> > in
>> > probability as we actually observe it more times??
>> >
>> > James
>> > --
>> > http://www-etud.iro.umontreal.ca/~bergstrj
>> >
>> > _______________________________________________
>> > SciPy-Dev mailing list
>> > SciPy-Dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>> >
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
>
> --
> http://www-etud.iro.umontreal.ca/~bergstrj
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>