[SciPy-user] Statistics advise with scipy

[David Huard]

I've had some success with the following:

1. Define a simple statistical model for your data.  That is, from the
previous data, define a distribution for the probability of the next
point.
2. Define a cutoff probability separating valid data from outliers.
3. For each datum, compute its probability based on previous data, and
tag it as valid or outlier.

The advantage is that you can start with a simple statistical model (
for example a gaussian centered on the last valid entry ) and
customize it as you find cases that are not well handled.

David

2008/7/22 didier rano <didier.rano at gmail.com>:
> Hi,
> I haven't found yet a solution to my problem. But I am reading a good
> article about removing
> outliers: http://www.lcgceurope.com/lcgceurope/data/articlestandard//lcgceurope/502001/4509/article.pdf
> Now, I need to experiment methods described in this article.
> Thanks
> Didier Rano
>
> 2008/7/22 Tim Michelsen <timmichelsen at gmx-topmail.de>:
>>
>> >> My data is not normal. Do you know robusts method in scipy ? Or maybe
>> >> in an
>> >> other python module ?
>> >
>> > Mmh, I'm sure you could implement some yourself. That way, we could
>> > start
>> > another scikits. There are already some winsorization and trimming
>> > functions
>> > in scipy.stats.
>> > Alternatively, you can try to use R and numpy through rpy:
>> > http://rpy.sourceforge.net/
>> Dider,
>> may I ask you to give some feedback what method worked for you?
>> I am also working with the problem of removing outliners etc. from data.
>>
>> Thanks in advance,
>> Timmie
>>
>> _______________________________________________
>> SciPy-user mailing list
>> SciPy-user at scipy.org
>> http://projects.scipy.org/mailman/listinfo/scipy-user
>
>
>
> --
> Didier Rano
> didier.rano at gmail.com
> http://www.jaxtr.com/didierrano
> _______________________________________________
> SciPy-user mailing list
> SciPy-user at scipy.org
> http://projects.scipy.org/mailman/listinfo/scipy-user
>
>