[SciPy-Dev] Suggest addition to scipy.stats : Mann-Kendall test

[josef.pktd at gmail.com]

On Tue, Jul 19, 2016 at 2:50 AM, Schultz, Martin
<m.schultz at fz-juelich.de> wrote:
> Hi Pauli,
>
>     thank you for the feedback. I would be willing to inquire with the original author(s) about licensing, and I can provide a few small test datasets with results for inclusion into an automated test, but I don't think I would be able to write the automated test myself. So, if someone volunteers on that end, I will communicate the data with her or him. May I get back to you in case of questions concerning the license terms etc. if needed?
>
> Martin Schultz
>
>
> -----Original Message-----
> From: SciPy-Dev [mailto:scipy-dev-bounces at scipy.org] On Behalf Of Pauli Virtanen
> Sent: Tuesday, July 19, 2016 1:30 AM
> To: scipy-dev at scipy.org
> Subject: Re: [SciPy-Dev] Suggest addition to scipy.stats : Mann-Kendall test
>
> Hi,
>
> Mon, 18 Jul 2016 10:11:31 +0000, Schultz, Martin kirjoitti:
>>      the routine below evaluates the Mann-Kendall test for
>>      non-parametrically checking the significance of any trend estimate.
>>      It would provide a nice complement to the existing theilslopes
>>      routine and others in scipy.stats. I found this code when searching
>>      for "Mann Kendall python" and saw that the original link was no
>>      longer existing. In my opinion it would be good to preserve this
>>      piece of work and make it available to others. I tested this
>>      routine on hundreds of datasets and it seemed to work well. This
>>      implementation was also compared to a few calculations with the
>>      Matlab implementation of this test and provided identical results.
>
> Thanks for the information and code.
>
> Note that the following technical points would need to be
> addressed for  including such a routine in Scipy:
>
> - No license information, so the code or works derived from it
>   cannot be used.
>
> - Tests. While checking the results manually is valuable,
>   it would also be necessary to write automated tests ---
>   to make it possible for other people to verify the correctness,
>   and to ensure the code stays working in the future.
>
> If you do not have time for this yourself, getting it done would need
> someone else to volunteer.

The function looks interesting, I've never heard of this hypothesis test.

The implementation in the function is very straightforward and I
expect that it will be pretty slow for longer arrays, especially when
there are just a few ties.
Maybe it is possible to streamline the function along some of the
existing functions. I don't know if it can be based on ranked data,
which IIRC are already in cython.
(shapiro which is another all pair comparison algorithm is in Fortran, IIRC)


(aside: the vsp package in the docstring looks interesting, and might
have a few more things that are missing in scipy.stats/statsmodels.)

Josef

>
> --
> Pauli Virtanen
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
> ------------------------------------------------------------------------------------------------
> ------------------------------------------------------------------------------------------------
> Forschungszentrum Juelich GmbH
> 52425 Juelich
> Sitz der Gesellschaft: Juelich
> Eingetragen im Handelsregister des Amtsgerichts Dueren Nr. HR B 3498
> Vorsitzender des Aufsichtsrats: MinDir Dr. Karl Eugen Huthmacher
> Geschaeftsfuehrung: Prof. Dr.-Ing. Wolfgang Marquardt (Vorsitzender),
> Karsten Beneke (stellv. Vorsitzender), Prof. Dr.-Ing. Harald Bolt,
> Prof. Dr. Sebastian M. Schmidt
> ------------------------------------------------------------------------------------------------
> ------------------------------------------------------------------------------------------------
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> https://mail.scipy.org/mailman/listinfo/scipy-dev