[SciPy-Dev] Suggest addition to scipy.stats : Mann-Kendall test

[josef.pktd at gmail.com]

On Tue, Jul 19, 2016 at 4:21 AM, Pierre GM <pgmdevlist at gmail.com> wrote:
> Just so you know, there's a Kendall's tau routine in scipy.mstats.

the kendall_tau in scipy.stats has been rewritten with a mix of numpy
and cython, and, based on how far I followed the PR, has very good
performance now.
I don't know if anything can be reused because the hypothesis is
pretty different, sign instead of correlation and univariate instead
of bivariate.

Still only mstats has the seasonal version of kendall_tau.
vsp package also has a seasonal version of the mann-kendall trend test

Josef

>
>
> On 19 July 2016 at 10:04:48, josef.pktd at gmail.com (josef.pktd at gmail.com)
> wrote:
>
> On Tue, Jul 19, 2016 at 2:50 AM, Schultz, Martin
> <m.schultz at fz-juelich.de> wrote:
>> Hi Pauli,
>>
>> thank you for the feedback. I would be willing to inquire with the
>> original author(s) about licensing, and I can provide a few small test
>> datasets with results for inclusion into an automated test, but I don't
>> think I would be able to write the automated test myself. So, if someone
>> volunteers on that end, I will communicate the data with her or him. May I
>> get back to you in case of questions concerning the license terms etc. if
>> needed?
>>
>> Martin Schultz
>>
>>
>> -----Original Message-----
>> From: SciPy-Dev [mailto:scipy-dev-bounces at scipy.org] On Behalf Of Pauli
>> Virtanen
>> Sent: Tuesday, July 19, 2016 1:30 AM
>> To: scipy-dev at scipy.org
>> Subject: Re: [SciPy-Dev] Suggest addition to scipy.stats : Mann-Kendall
>> test
>>
>> Hi,
>>
>> Mon, 18 Jul 2016 10:11:31 +0000, Schultz, Martin kirjoitti:
>>> the routine below evaluates the Mann-Kendall test for
>>> non-parametrically checking the significance of any trend estimate.
>>> It would provide a nice complement to the existing theilslopes
>>> routine and others in scipy.stats. I found this code when searching
>>> for "Mann Kendall python" and saw that the original link was no
>>> longer existing. In my opinion it would be good to preserve this
>>> piece of work and make it available to others. I tested this
>>> routine on hundreds of datasets and it seemed to work well. This
>>> implementation was also compared to a few calculations with the
>>> Matlab implementation of this test and provided identical results.
>>
>> Thanks for the information and code.
>>
>> Note that the following technical points would need to be
>> addressed for including such a routine in Scipy:
>>
>> - No license information, so the code or works derived from it
>> cannot be used.
>>
>> - Tests. While checking the results manually is valuable,
>> it would also be necessary to write automated tests ---
>> to make it possible for other people to verify the correctness,
>> and to ensure the code stays working in the future.
>>
>> If you do not have time for this yourself, getting it done would need
>> someone else to volunteer.
>
> The function looks interesting, I've never heard of this hypothesis test.
>
> The implementation in the function is very straightforward and I
> expect that it will be pretty slow for longer arrays, especially when
> there are just a few ties.
> Maybe it is possible to streamline the function along some of the
> existing functions. I don't know if it can be based on ranked data,
> which IIRC are already in cython.
> (shapiro which is another all pair comparison algorithm is in Fortran, IIRC)
>
>
> (aside: the vsp package in the docstring looks interesting, and might
> have a few more things that are missing in scipy.stats/statsmodels.)
>
> Josef
>
>>
>> --
>> Pauli Virtanen
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>>
>> ------------------------------------------------------------------------------------------------
>>
>> ------------------------------------------------------------------------------------------------
>> Forschungszentrum Juelich GmbH
>> 52425 Juelich
>> Sitz der Gesellschaft: Juelich
>> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
>> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
>> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
>> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
>> Prof. Dr. Sebastian M. Schmidt
>>
>> ------------------------------------------------------------------------------------------------
>>
>> ------------------------------------------------------------------------------------------------
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> https://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev