[SciPy-Dev] [scipy/scipy] ENH: Add PDF, CDF and parameter estimation for Stable Distributions (#7374)

Sun Apr 8 13:05:07 EDT 2018

> On 8 April 2018 at 12:49, Andrea Karlova <andrea.karlova at gmail.com> wrote:
>> On 8 April 2018 at 05:21, Blair Azzopardi <blairuk at gmail.com> wrote:
>>> On 8 April 2018 at 01:49, Andrea Karlova <andrea.karlova at gmail.com>
wrote:
>>>> On 28 November 2017 at 18:33, Blair Azzopardi <blairuk at gmail.com>
wrote:
>>>>> On Tue, 28 Nov 2017, 16:01 Andrea Karlova, <andrea.karlova at gmail.com>
wrote:
>>>>>> On 28 November 2017 at 13:17, Blair Azzopardi <blairuk at gmail.com>
wrote:
>>>>>>> ---------- Forwarded message ---------
>>>>>>> From: An <notifications at github.com>
>>>>>>> Date: Tue, 28 Nov 2017, 10:00
>>>>>>> Subject: Re: [scipy/scipy] ENH: Add PDF, CDF and parameter
estimation for Stable Distributions (#7374)
>>>>>>> To: scipy/scipy <scipy at noreply.github.com>
>>>>>>>
>>>>>>> Blair, is there a way to have a chat via email?
>>>>>>>
>>>>>>> On 24 November 2017 at 20:38, Blair Azzopardi <
notifications at github.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>> Hi @an81 <https://github.com/an81>. Thank you for the 550+ page
book.
>>>>>>> Please can you be a bit more specific? Some sample code goes a long
way
>>>>>>> too. Also can you perhaps test the existing code and highlight
where the
>>>>>>> Gibbs effect might be more prominent? eg low alpha etc; perhaps
this can be
>>>>>>> just documented with a recommendation that users use quad in these
cases
>>>>>>> (already in code). This is until better implementation is available.
>>>>>>> —
>>>>>>> You are receiving this because you were mentioned.
>>>>>>> Reply to this email directly, view it on GitHub
>>>>>>> <https://github.com/scipy/scipy/pull/7374#issuecomment-346893569>,
or mute
>>>>>>> the thread
>>>>>>> <
https://github.com/notifications/unsubscribe-auth/AIfE8VNjHdqANvYfUG8Gg6feKb5np_kLks5s5ylhgaJpZM4NQBiP
>
>>>>>>
>>>>>> Hi Andrea
>>>>>>
>>>>>> I hope you're well and yes no problem talking chatting via email.
Actually makes sense.
>>>>>>
>>>>>> Also, I hope you don't mind me messaging you via an email address
found in a previous comment.
>>>>>>
>>>>>> I took a look at that book you linked to but unfortunately I don't
have enough time to process it currently. I will read it in time mind you.
I've found a shorter paper that might offer similar suggestions to yours:
>>>>>>
>>>>>>
http://prac.im.pwr.edu.pl/~hugo/publ/SFB2005-008_Borak_Haerdle_Weron.pdf
>>>>>>
>>>>>> Although I'm not 100% sure as it doesn't mention mejer g functions.
What are your thoughts?
>>>>>>
>>>>>> Kind regards
>>>>>> Blair
>>>>>
>>>>> Hi Blair,
>>>>>
>>>>> thats great.
>>>>> Thx for your email.
>>>>>
>>>>> I ll give you access to my Dropbox folder where I have materials
relevant to stable laws.
>>>>> For working with stable laws I found really useful to understand
>>>>> and actively switch between different parametrizations of the
characteristic exponent.
>>>>> Zolotarev and others would have polenty of parametrization and each
of them is helpful
>>>>> for different task.
>>>>>
>>>>> I guess I can share with you my python  code on the github,
>>>>> you can contribute to it if you would feel like so
>>>>> and then we can just plug it into scipy lib.
>>>>>
>>>>> Is it ok to use this gmail account for sending you invitation to
Dropbox?
>>>>>
>>>>> Thx,
>>>>> Kind regards,
>>>>>
>>>>> Andrea
>>>>
>>>> Hi Andrea
>>>>
>>>> Yes, please do share with this email address.
>>>>
>>>> Is it possible you could commit your changes to the existing PR I've
already set up? I believe this is possible by forking my repo (script fork)
and committing to that. The PR includes parameter estimation and some
general framework changes around this distribution too. Also I feel there
are still use cases where it's useful keep fft method. I've added you a
collaborator on my fork.
>>>>
>>>> https://github.com/bsdz/scipy
>>>>
>>>> Kind regards
>>>> Blair
>>>
>>> Hi Blair,
>>>
>>> hope you are well.
>>> I was looking into your code on stable laws in scipy.
>>> I ll be running a group of 4 people at hackaton which is  organized by
AHL in 2 weeks time
>>> and I am planning to revise and add more code into stable laws
implementation.
>>>
>>> I was wondering if you can give me some quick update on what is done so
far
>>> and what are urgent issues at this point according to your opinion?
>>> I have some ideas what I would like add, also the documentation page
needs to be written,
>>> but just a quick check on what methods you implemented for calculating
of
>>> pdfs, cdfs, and parameters estimates and how it went?
>>>
>>> Thanks a lot.
>>> Kind regards,
>>> Andrea
>>>
>>
>> Hi Andrea
>>
>> Thanks for your email. I'm cc-ing this email along with out previous
correspondence to the scipy-dev mailing list (as bottom post) . Please
direct all your future messages here.
>>
>> It's good to hear you plan to improve my PR (coincidentally at my old
employer Man, although at the time they weren't interested in Stable laws).
>>
>> I did implement your suggestions to use Zolotarev's formulations. All
the code is under https://github.com/scipy/scipy/pull/7374 as you are aware.
>>
>> You could perhaps look at improving documentation, adding more /
improving tests and any optimisations you can think of.
>>
>> Either I can re-add you as a collaborator to my repo (I removed you when
I hadn't heard back from you previously) or you can email me a patch and
I'll integrate it into PR 7374.
>>
>> Thanks
>> Blair
>
> Hi Blair,
>
> yes, if you could add me to your repository,
> so I can look into what is done and start to test your code,
> so that I can look into some patches, that would be great.

Hi Andrea

Actually, just to be cautious. Could you fork my scipy repo (it has all the
changes there) and then create a PR into that repo? I can then merge it and
it should push upstream into my main PR.

> How does your code performs in neighbourhood of alpha = 1?
> How does it perform when the asymetry parameter beta gets close to its
boundary values?

Please run the code and check. My testing took samples from Nolan's public
domain executables and tested against those values. Although I tested
against a range of parameters I might not have tested those ranges
specifically. It's all in the unit tests in the PR, see test class
TestLevyStable. Extending the testing here would be helpful.

> Do you have somewhere in Latex notes on your fft implementation,
> with some theoretical investigation of the performance of the method?

This is all in the code documentation along with references.

> I am planning to write a research paper about the implementation in the
Scipy, which I ll put on my ArXiv,
> and to which I ll refer in the documentation, so that if people are
interested in the performance, they can look into paper.
> So I ll be listing the co-authors of this piece of code there.
> If there are other people who worked on the stable laws implementation in
Scipy, it would be nice if I can have the list of them.
> Also if you have some note on fft,  I can join it into my research note
and add you as  a coauthor of the note,
> depends upon you.

So far, I'm the sole author of this PR. Again, all papers and references
are in the PR. Please please do look at the code.

>
> Kind regards,
> Andrea

Just some additional points, you need to subscribe to the scipy-dev mailing
list for your messages to get through. Also, the mailing list uses bottom
posting for email messages; that means, all replies are bottom of original
message (not the top.)

Look forward to your contribution - thanks!

Blair
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20180408/ae823eaf/attachment.html>