[SciPy-Dev] Regarding taking up project ideas and GSoC 2015
Ralf Gommers
ralf.gommers at gmail.com
Sun Mar 15 12:44:21 EDT 2015
On Sat, Mar 14, 2015 at 4:53 PM, Maniteja Nandana <
maniteja.modesty067 at gmail.com> wrote:
> Hi everyone,
>
> I was hoping if I could get some suggestions regarding the API for
> *scipy.diff* package.
>
> 1. Type of input to be given - callable function objects or a set of
> points as in scipy.integrate.
>
> I would expect functions.
>
> 1. Parameters to be given to derivative methods, like *method *(as in
> scipy.optimize) to accommodate options like *central, forward,
> backward, complex or richardson.*
>
> There may be a lot of parameters that make sense, depending on the exact
differentiation method(s) used. I think it's important to think about which
ones will be used regularly, and which are only for niche usecases or power
users that really understand the methods. Limit the number of parameters,
and provide some kind of configuration object to tweak detailed behavior.
This is the constructor of numdifftools.Derivative, as an example of too
many params:
def __init__(self, fun, n=1, order=2, method='central', romberg_terms=2,
step_max=2.0, step_nom=None, step_ratio=2.0, step_num=26,
delta=None, vectorized=False, verbose=False,
use_dea=True):
> 1. The maximum order of derivative needed ? Also the values of order
> *k* used in the basic method to determine the truncation error O(h^k) ?
> 2. API defined in terms of functions(as in statsmodels) or classes(as
> in numdifftools) ?
>
> No strong preference, as long as it's a coherent API. The scipy.optimize
API (minimize, root) is nice, something similar but as classes is also fine.
>
> 1. Return type of the methods should contain the details of the
> result, like *error *?( on lines of OptimizeResult, as
> in scipy.optimize )
>
> I do have a strong preference for a Results object where the number of
return values can be changed later on without breaking backwards
compatibility.
I would really appreciate some feedback and suggestions on these issues.
> The whole draft of the proposal can be seen here
> <https://github.com/maniteja123/GSoC/wiki/Proposal%3A-add-finite-difference-numerical-derivatives-as-%60%60scipy.diff%60%60>
> .
>
Regarding your "to be discussed" list:
- Don't worry about the module name (diff/derivative/...), this can be
changed easily later on.
- Broadcasting: I'm not sure what needs to be broadcasted. If you provide a
function and the derivative order as int, that seems OK to me.
- Parallel evaluation should be out of scope imho.
Cheers,
Ralf
> Thanks for reading along and giving your valuable inputs.
>
> Cheers,
> Maniteja.
>
> On Wed, Mar 11, 2015 at 11:44 PM, Maniteja Nandana <
> maniteja.modesty067 at gmail.com> wrote:
>
>> Hi everyone,
>>
>> I have created a Wiki <https://github.com/maniteja123/GSoC/wiki/> page
>> and draft proposal
>> <https://github.com/maniteja123/GSoC/wiki/Proposal:-add-finite-difference-numerical-derivatives-as-%60%60scipy.diff%60%60> regarding
>> some approaches for API implementation for scipy.diff package after
>> discussing with Christoph Deil. I would really appreciate some feedback and
>> suggestions to incorporate more sound and concrete ideas into the proposal.
>> I also wanted to ask if it would be better to start a wiki page regarding
>> this on scipy repository. I thought it would be better to do so once the
>> proposal is more concrete.
>>
>> Thanks again for reading along my proposal and waiting in anticipation
>> for your suggestions.
>>
>> Cheers,
>> Maniteja
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>> On Mon, Mar 9, 2015 at 12:23 PM, Ralf Gommers <ralf.gommers at gmail.com>
>> wrote:
>>
>>> Hi Maniteja,
>>>
>>>
>>> On Fri, Mar 6, 2015 at 1:12 PM, Maniteja Nandana <
>>> maniteja.modesty067 at gmail.com> wrote:
>>>
>>>> Hello everyone,
>>>>
>>>> I am writing this mail to enquire about implementing numerical
>>>> differentiation package in scipy.
>>>>
>>>> There have been discussions before (Issue #2035
>>>> <https://github.com/scipy/scipy/issues/2035>) and some PRs (PR #2835
>>>> <https://github.com/scipy/scipy/pull/2835>) to include tools to
>>>> compute derivatives in scipy.
>>>>
>>>> According to the comments made in them, as far as I can understand, I
>>>> see that there are some ways to do derivatives on the computer with varying
>>>> generality and accuracy ( by Alex Griffing
>>>> <https://github.com/scipy/scipy/issues/2035#issuecomment-23628615> ) :
>>>>
>>>> 1. 1st order derivatives of special functions
>>>> 2. derivatives of univariate functions
>>>> 3. symbolic differentiation
>>>> 4. numerical derivatives - finite differences
>>>> 5. automatic or algorithmic differentiation
>>>>
>>>> Clearly, as suggested in the thread, the 1st option is already done in
>>>> functions like *jv* and *jvp* in *scipy.special. *
>>>>
>>>> I think everyone agreed that symbolic derivatives is out of scope of
>>>> scipy.
>>>>
>>>
>>> Definitely, symbolic anything is out of scipy for scipy:)
>>>
>>>
>>>> Though I would like to hear more about the univariate functions.
>>>>
>>>> Coming to finite differences, the modules described there, *statsmodels
>>>> *and *numdifftools, *they vary in aspects of speed and accuracy, in
>>>> terms of approaches followed as mentioned in Joseph Perktold comment
>>>> <https://github.com/scipy/scipy/pull/2835#issuecomment-52372036>
>>>>
>>>> - *Statsmodels *used complex step derivatives, which are for first
>>>> order derivatives and have only truncation error, no roundoff error
>>>> since there is no subtraction.
>>>> - *Numdifftools *uses adaptive step-size to calculate finite
>>>> differences, but will suffer from dilemma to choose small step-size to
>>>> reduce truncation error but at the same time avoid subtractive cancellation
>>>> at too small values
>>>>
>>>> I have read the papers used by both the implementations:
>>>> *Statsmodels *Statistical applications of the complex-step method of
>>>> numerical differentiation, Ridout, M.S.
>>>> <https://drive.google.com/file/d/0BwUeCS0FJLRucXdCc1JOTEY0cGc/view?usp=sharing>
>>>> *Numdifftools *The pdf attached in the github repository DERIVEST.pdf
>>>> <https://drive.google.com/file/d/0BwUeCS0FJLRuYW1MNlp2enJCaHM/view?usp=sharing>
>>>>
>>>> Just pointing out in this platform, I think there is an error in
>>>> equation 13 in DERIVEST, It should be
>>>>
>>>> f'-0() = 2f'-delta/2() - f'-delta(), instead of f'-0() = 2f'-delta()
>>>> - f'-delta/2()
>>>>
>>>> as also correctly mentioned in the matlab code that followed the
>>>> equation
>>>>
>>>
>>> You may want to let the author know, he'll probably appreciate it.
>>>
>>>
>>>> As much as my understanding from the discussions goes, the statsmodels
>>>> implementation uses elegant broadcasting. Though I get the idea seeing the
>>>> code, I would really appreciate some examples that clearly explain this.
>>>>
>>>> Also the complex-step method is only for first order derivatives and
>>>> that function is analytic, so that Cauchy-Riemann equations are valid. So,
>>>> is it possible to differentiate any function with this ?
>>>>
>>>> Also as I was discussing with Christoph Deil, the API implementation
>>>> issue of whether to use classes, as in numdifftools or as functions, as in
>>>> statsmodels came to the fore. Though I am not an expert in it, I would love
>>>> to hear some suggestions on it.
>>>>
>>>
>>> It will be important to settle on a clean API. There's no general
>>> preference for classes or functions in Scipy, the pros/cons have to be
>>> looked at in detail for this functionality. The scope of the scipy.diff
>>> project is quite large, so starting a document (as I think you've already
>>> discussed with Christoph?) outlining the API that can be reviewed will be a
>>> lot more efficient than trying to do it by email alone.
>>>
>>>
>>>> Though at this point AD seems ahead of time, it is powerful in forward
>>>> and reverse methods, moreover complex-step is somewhat similar to it. The
>>>> packages *ad *and *algopy *use AD. Also, there were concerns with
>>>> interfacing these methods with C/ Fortran functions. It would also be
>>>> great if there could be suggestions regarding whether to implement these
>>>> methods.
>>>>
>>>
>>> It's been around for a while so not sure about "ahead of its time", but
>>> yes it can be powerful. It's a large topic though, should be out of scope
>>> for this GSoC project. Good finite difference methods will be challenging
>>> enough:) That doesn't mean that AD is out of scope for Scipy necessarily,
>>> but that's for another time to discuss.
>>>
>>> At the same time, it would be really helpful if any new methods or
>>>> packages to be looked into could be suggested.
>>>>
>>>
>>> I think what's in numdifftools and statsmodels is a good base to build
>>> on. What could be very useful in addition though is an indepent reference
>>> implementation of the methods you're working on. This could be
>>> Matlab/R/Julia functions or some package written by the author of a paper
>>> you're using. I don't have concrete suggestions now - you have a large
>>> collection of papers - but you could already check the papers you're using.
>>>
>>> Cheers,
>>> Ralf
>>>
>>>
>>>> Waiting in anticipation for your feedback and response. Happy to learn
>>>> :)
>>>> Thanks for reading along my lengthy mail. Please do correct if I did
>>>> some mistake.
>>>>
>>>> I have attached the documents I have related to these issues, most
>>>> importantly *The Complex-Step Derivative Approximation by **JOAQUIM R.
>>>> R. A. MARTINS*
>>>>
>>>> *Numerical differentiation
>>>> <https://drive.google.com/folderview?id=0BwUeCS0FJLRufnJaVko3MGpJX0Nud3R0dHgyc2JBYUgxVkhBTkNvbkhFQWZucmhWSzlaVVk&usp=sharing>*
>>>>
>>>> Cheers,
>>>> Maniteja.
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at scipy.org
>>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>>
>>>
>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>>
>>
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150315/bc10f3ea/attachment.html>
More information about the SciPy-Dev
mailing list