[SciPy-Dev] New documentation section: "SciPy API Development Guide"

[Warren Weckesser]

On 5/11/20, rlucas7 at vt.edu <rlucas7 at vt.edu> wrote:
> One thing that’s come up is whether the rv_discrete should have a fit
> method?
>
> I think I’d called this out awhile back, seems other’s noticed too:
>
> https://github.com/scipy/scipy/issues/11948#issuecomment-620308082
>
> I mentioned to submit the question here but I didn’t see, maybe I missed.
>
> That is the largest difference in terms of api between discrete and
> continuous distributions to my knowledge in SciPy.
>
> It sounds like these are different. One talks about subclassing, E.g using
> existing framework. What I understood Warren was referring to was feature
> development of the infrastructure itself.
>
> Maybe I misunderstood?


Lucas,

Just getting back to this email thread--sorry for letting it hang for a while...

Adding a fit method for the discrete distributions would be a great
enhancement.  But what I'm talking about here is documenting the
existing infrastructure and conventions that we currently use.  How
does the enhancement request for a fit method for discrete
distribution connect to what I am proposing here?   Once the
enhancement has been added, we ensure that the procedure for
implementing the method in a new subclass is thoroughly documented.
That is, what method do I have to override, and what is the API that
my method must satisfy.

Warren


>
> Sincerely,
>
> -Lucas Roberts
>
>>> On May 11, 2020, at 3:49 PM, Warren Weckesser
>>> <warren.weckesser at gmail.com> wrote:
>>>
>>> On 5/11/20, Paul van Mulbregt <p.vanmulbregt at comcast.net> wrote:
>>> Hi Warren,
>>> You mentioned "Creating probability distributions in stats: a guide to
>>> subclassing rv_continuous and rv_discrete."
>>> There is already some existing doc on adding distributions
>>> https://docs.scipy.org/doc/scipy/reference/dev/contributor/adding_new.html#adding-new
>>> (reachable from
>>> https://docs.scipy.org/doc/scipy/reference/dev/contributor/contributor_toc.html.)
>>> How would the two relate to each other?
>>
>>
>> Heh, maybe try to make that doc easier to find?
>>
>> Then add more specific details about the required API that must be
>> implemented.  Two items that come to mind: explain the required
>> signature of `_rvs()`, and what values will be passed to it from
>> `rvs()`; and explain the requirements for implementing broadcasting in
>> the overridden functions.  We can add documentation for other "pain
>> points" as we encounter (or remember) them.
>>
>> Warren
>>
>>
>>
>>
>>
>>> -Paul
>>>> On May 9, 2020, at 10:04 AM, Warren Weckesser
>>>> <warren.weckesser at gmail.com>
>>>> wrote:
>>>> On 5/9/20, Ralf Gommers <ralf.gommers at gmail.com
>>>> <mailto:ralf.gommers at gmail.com>> wrote:
>>>>> On Sat, May 9, 2020 at 5:43 AM Warren Weckesser
>>>>> <warren.weckesser at gmail.com>
>>>>> wrote:
>>>>>> Hey all,
>>>>>> I just created a pull request in which I started a new subsection of
>>>>>> the developer documentation called the "SciPy API Development Guide".
>>>>>> As I write this, the CircleCI artifact containing the new page is
>>>>>> https://19020-1460385-gh.circle-artifacts.com/0/html-scipyorg/dev/api-dev/api-dev-toc.html
>>>>> That looks like a good idea to me, thanks Warren. So far this lived at
>>>>> the
>>>>> bottom of the core developer guide:
>>>>> http://scipy.github.io/devdocs/dev/core-dev/index.html#module-specific-instructions.
>>>>> Could you move the scipy.special docs there over to this new API
>>>>> Development Guide?
>>>> Good idea.  I just added a commit to make that change in the pull
>>>> request.
>>>>> The first item to be included in this section is a design
>>>>>> specification for the `nan_policy` argument that is used in many
>>>>>> functions in `scipy.stats`.  We'll be adding `nan_policy` to more
>>>>>> functions, so I think it will be helpful to have an explicit design
>>>>>> spec in the docs that we all agree on.
>>>>> +1 that would be super helpful.
>>>>>> Two more topics to be added to the guidelines soon:
>>>>>> * Creating probability distributions in stats: a guide to subclassing
>>>>>> rv_continuous and rv_discrete.
>>>>>> * Modifying return values to allow objects with attributes while
>>>>>> maintaining backward compatibility with existing return values that
>>>>>> are tuples or NamedTuples.
>>>>> Both of these also sound very useful. For namedtuples, you probably
>>>>> should
>>>>> also include or at least reference the Bunch design pattern (we have
>>>>> it
>>>>> in
>>>>> `integrate` and `signal`, there's still an open issue for `stats`).
>>>> Yes, that's what I had in mind.
>>>> Warren
>>>>> Cheers,
>>>>> Ralf
>>>>>> I'm currently focused on `stats`, but this section is meant for any
>>>>>> API guidelines or conventions that we use but that are currently only
>>>>>> stored in our heads, in old mailing list or github threads, or in a
>>>>>> few implementations here and there in the code.
>>>>>> Take a look and let me know what you think.
>>>>>> Warren
>>>>>> _______________________________________________
>>>>>> SciPy-Dev mailing list
>>>>>> SciPy-Dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>