[SciPy-Dev] GSoC 2019

Tue Apr 2 02:20:49 EDT 2019

Hey,
Greg, I went through the ideas you have mentioned and also looked up the
matplotlib docs as suggested.  And I feel like creating a new namespace
would be a better option as it won't require many changes in the current
implementation. As matplotlib.pyplot is a well-implemented module  I feel
like we can consider it as our basic structure and start implementing the
backend.

For the start, I think import can look like this(rough idea)
import scipy.fftpack.pyfftw as fft
where pyffw can be replaced by any third party fftpack.

Cheers.
-Gopi

On Sun, Mar 31, 2019 at 2:00 AM Gopi Manohar Tatiraju <deathcoderx at gmail.com>
wrote:

> Hey,
> I feel like I need to make some more changes regarding the idea. I went
> through your reply to Anany and I will consider all those points and will
> get back to you once I come up with something.
>
> -Gopi.
>
> On Sun, Mar 31, 2019 at 5:24 AM Gregory Lee <grlee77 at gmail.com> wrote:
>
>> Hi Gopi,
>>
>> I am not sure I follow exactly what you are suggesting here. Are you
>> proposing that the `fftpackN` variable in your example would be a Python
>> module corresponding to a given backend? If so, why is an array of data, z,
>> being passed into the newAPI function here? I don't see why a data array
>> should be needed at all to choose a backend.
>>
>> Please also see my response to Anany's recent post on this list for a
>> couple of other suggestions of things to consider in terms of the API.
>>
>> - Greg
>>
>>
>> On Fri, Mar 29, 2019 at 5:27 AM Gopi Manohar Tatiraju <
>> deathcoderx at gmail.com> wrote:
>>
>>> Hey,
>>> I went through the code and got some ideas.
>>> So what will happen is:
>>> First, we will declare FFTW as we use them normally.
>>>
>>> z = np.random.rand(2, 128, 64)
>>> pyfftw.FFTW(z, axes=(-1, ))
>>> ..
>>> ..
>>> Now after declaring our basic work to shift to another FFTpack, what we
>>> can do is:
>>> fftPackN= newAPI(z, pack = 'PYFFTW');
>>>
>>> Pack will take the name of the 3rd party FFT pack which we want to use.
>>> Suggestions needed.
>>>
>>> Cheers.
>>>
>>>
>>
>>>
>>> On Wed, Mar 27, 2019 at 8:43 PM Eric Larson <larson.eric.d at gmail.com>
>>> wrote:
>>>
>>>> After reading the related issues and thinking about the problem,
>>>> perhaps the best first step is you come up with some initial API proposal.
>>>>
>>>> One way to work through the process is to think about the use cases
>>>> that we want to support. How should users actually write code to use the
>>>> new functionality? You can start with short code snippets that do not work
>>>> currently, but should work for users at the end of the project. Then think
>>>> about what needs to change inside SciPy for these code snippets to work.
>>>> This can become an iterative process as you realize that the SciPy changes
>>>> might be better or easier with changes to the user-facing API, etc.
>>>>
>>>> Eric
>>>>
>>>>
>>>> On Tue, Mar 26, 2019 at 3:38 PM Gopi Manohar Tatiraju <
>>>> deathcoderx at gmail.com> wrote:
>>>>
>>>>> Hey,
>>>>> As proposal submission period started I want to discuss a bit anobt
>>>>> how can we implement the API.
>>>>> I would very much appriciate inputs and guidance.
>>>>>
>>>>> Cheers.
>>>>>
>>>>> On Thu, Mar 7, 2019 at 11:40 PM Eric Larson <larson.eric.d at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> The best way to figure that out is probably to take a look at some of
>>>>>> the existing (minor) SciPy GitHub issues and try to work on one or two that
>>>>>> you feel like working on. Actually trying to work out how to contribute
>>>>>> code changes (as independently as possible/reasonable based on the existing
>>>>>> guides/docs already discoverable online) will quickly highlight things that
>>>>>> need to be learned. Some of the general things are the SciPy contributing
>>>>>> guidelines, standard GitHub PR workflow, communication with reviewers, and
>>>>>> so forth.
>>>>>>
>>>>>> For this particular project, it's important to understand the various
>>>>>> flavors and properties of FFT in SciPy (real, complex, n-dimensional,
>>>>>> caching plans, etc.) as well as other projects like NumPy, CuPy, and
>>>>>> pyFFTW. Some of this can be learned and improved upon as part of GSoC, but
>>>>>> the more you know and can demonstrate / put into practice knowledge of
>>>>>> before the applications are reviewed, the better.
>>>>>>
>>>>>> Eric
>>>>>>
>>>>>>
>>>>>> On Wed, Mar 6, 2019 at 4:55 AM Gopi Manohar Tatiraju <
>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>
>>>>>>> Thank you for the detailed reply.
>>>>>>> I will keep these point in mind while preparing my proposal. Apart
>>>>>>> from proposal, what skills shall I improve so that I can contribute more
>>>>>>> effectively and efficiently.
>>>>>>>
>>>>>>> Cheers.
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Mar 6, 2019, 3:42 AM Eric Larson <larson.eric.d at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> There are a number of considerations that would be good to address.
>>>>>>>> I'd start by considering the things that are typically considered
>>>>>>>> when making changes to SciPy
>>>>>>>> <https://docs.scipy.org/doc/scipy/reference/hacking.html#contributing-to-scipy>,
>>>>>>>> but I'd expand it to a longer list since the scope is bigger than a typical
>>>>>>>> single PR. The list would contain (at least):
>>>>>>>>
>>>>>>>> - what will the user-facing API / contract be?
>>>>>>>> - what in SciPy needs to change to allow it (internals /
>>>>>>>> implementation outline, perhaps skeleton functions/classes)?
>>>>>>>> - what examples should we show users (in general this ends up being
>>>>>>>> just simple use cases of the API)?
>>>>>>>> - how will we test it for correctness?
>>>>>>>> - how will we benchmark it to prevent performance regressions?
>>>>>>>> - how can we break up the work for these steps (if possible) into
>>>>>>>> separable, sequential (or -- better but not usually possible --
>>>>>>>> independent) PRs?
>>>>>>>>
>>>>>>>> You might notice that none of these points are actually all that
>>>>>>>> specific to the FFT project, but can be generally applicable to GSoC
>>>>>>>> projects. For the FFT project, I would say "how do we build the backend" is
>>>>>>>> indeed implicitly contained / outlined in these bullet points. And as with
>>>>>>>> most projects, the first two points (and possibly the last) will probably
>>>>>>>> be the most challenging to sort out.
>>>>>>>>
>>>>>>>> Eric
>>>>>>>>
>>>>>>>>
>>>>>>>> On Mon, Mar 4, 2019 at 10:09 PM Gopi Manohar Tatiraju <
>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hey Eric,
>>>>>>>>> So for my proposal, shall I include how can we built a back end?
>>>>>>>>> What all points shall I include to make it effective?
>>>>>>>>>
>>>>>>>>> Cheers.
>>>>>>>>>
>>>>>>>>> On Tue, Mar 5, 2019, 4:50 AM Eric Larson <larson.eric.d at gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> That would be me -- don't worry, I'm on this list, as are
>>>>>>>>>> (presumably) other developers interested in the discussion, so no need to
>>>>>>>>>> document elsewhere for now from our end. It looks like Greg has summarized
>>>>>>>>>> the current status and thoughts we've had, so I don't have much to add at
>>>>>>>>>> this point.
>>>>>>>>>>
>>>>>>>>>> Eric
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sun, Mar 3, 2019 at 11:59 PM Gopi Manohar Tatiraju <
>>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hey Greg,
>>>>>>>>>>> I got the basic idea about what we want to implement this year.
>>>>>>>>>>> Our first priority is to create a backend for 3rd party
>>>>>>>>>>> fftpacks. We can create API using Python and Flask or any other suitable
>>>>>>>>>>> framework.
>>>>>>>>>>>
>>>>>>>>>>> I thought this discussion was going on on SciPy Developer List
>>>>>>>>>>> only. So do I need to document it all and post it somewhere so that other
>>>>>>>>>>> contributors can see this?
>>>>>>>>>>> Also, I couldn't find any contact info regarding Eric. It would
>>>>>>>>>>> be very helpful if we include him in this discussion as this is a roadmap
>>>>>>>>>>> for GSoC project and I can also get more insight into the project and
>>>>>>>>>>> expected outcomes.
>>>>>>>>>>>
>>>>>>>>>>> Cheers.
>>>>>>>>>>>
>>>>>>>>>>> On Mon, Mar 4, 2019 at 9:31 AM Gregory Lee <grlee77 at gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sun, Mar 3, 2019 at 12:28 PM Gopi Manohar Tatiraju <
>>>>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hey Gregory,
>>>>>>>>>>>>> I went through the idea and I am very much interested in
>>>>>>>>>>>>> working on it. I had Fourier Transforms as a part of my academic course, so
>>>>>>>>>>>>> I am well aware with basic concepts.
>>>>>>>>>>>>> One thing I want to discuss is, its mentioned that we have to
>>>>>>>>>>>>> create a backend system for 3rd party FFTpacks so my dobuts
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>> 1. How are we going to create API?
>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> This will needed to be decided on in consensus with the SciPy
>>>>>>>>>>>> developers and user community. I am mentoring for the project as a
>>>>>>>>>>>> contributor here and a maintainer of the related pyFFTW package, but I am
>>>>>>>>>>>> not a core SciPy dev (the other mentor, Eric, is a core dev here). I have
>>>>>>>>>>>> not discussed the API in any detail with the SciPy team at this stage.
>>>>>>>>>>>> Working out details of what the API should look like and working to
>>>>>>>>>>>> finalize that in collaboration with the community will be a primary task
>>>>>>>>>>>> during the early part of the GSOC project.
>>>>>>>>>>>>
>>>>>>>>>>>> 2. After the creation of backend will user at the time of
>>>>>>>>>>>>> installation of SciPy will decide what all 3rd party FFTpacks he needs or
>>>>>>>>>>>>> we will include some by default?
>>>>>>>>>>>>>
>>>>>>>>>>>>> The idea is to allow users to opt-in to a different FFT
>>>>>>>>>>>> backend at run time, but these alternative libraries will not be
>>>>>>>>>>>> distributed with SciPy itself due to incompatible software licenses (pyFFTW
>>>>>>>>>>>> and mkl-fft) or a requirement for specific hardware (CuPy requires an
>>>>>>>>>>>> NVIDIA GPU). SciPy will continue to offer its own bundled FFT library
>>>>>>>>>>>> (currently fftpack, but this could be switched to pocketfft as discussed in
>>>>>>>>>>>> the ideas page). The purpose of the backend interface is to provide a
>>>>>>>>>>>> convenient way for users to have third party libraries perform the FFTs
>>>>>>>>>>>> instead, but under the existing scipy.fftpack API. This is motivated by a
>>>>>>>>>>>> desire by end users and downstream projects to have features such as
>>>>>>>>>>>> multi-threading (pyFFTW and mkl-fft) or GPU support (CuPy) that are not
>>>>>>>>>>>> implemented in scipy.fftpack. Ultimately it will be up to users that opt-in
>>>>>>>>>>>> to using these other libraries to confirm that the license terms and are
>>>>>>>>>>>> compatible with their use case.
>>>>>>>>>>>>
>>>>>>>>>>>> p.s. It is not a big deal, but I think the SciPy-Dev mailing
>>>>>>>>>>>> lists prefers replies **below** the email you are replying to so people can
>>>>>>>>>>>> more easily follow the thread of discussion. There are some guidelines at:
>>>>>>>>>>>> https://www.scipy.org/scipylib/mailing-lists.html
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> I will research more about how to implement a backend myself
>>>>>>>>>>>>> and will let you know as soon as I come up with something new.
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>> CHeers,
>>>>>>>>>>>>> Greg
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sun, Mar 3, 2019 at 10:25 PM Gregory Lee <grlee77 at gmail.com>
>>>>>>>>>>>>> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sun, Mar 3, 2019 at 4:19 AM Gopi Manohar Tatiraju <
>>>>>>>>>>>>>> deathcoderx at gmail.com> wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> I want to work on SciPy in GSoC 2019. I started fixing some
>>>>>>>>>>>>>>> bugs, also made some pull request. Can anyone suggest me how should I
>>>>>>>>>>>>>>> proceed further?
>>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi Gopi,
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I think you have probably already found it, but the GSOC
>>>>>>>>>>>>>> project we have already identified mentors for in SciPy is here:
>>>>>>>>>>>>>> https://github.com/scipy/scipy/wiki/GSoC-2019-project-ideas
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If this is a project you are interested in then I would start
>>>>>>>>>>>>>> thinking about what the FFT interface would look like and reviewing some of
>>>>>>>>>>>>>> the past/present issues related to fftpack / FFT in SciPy. Please ask
>>>>>>>>>>>>>> questions you have related to that project here so that you can clarify
>>>>>>>>>>>>>> what would be involved in writing a successful application.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> If you have a different idea in mind you can propose it here,
>>>>>>>>>>>>>> but you should do that soon, because it will be necessary to determine if
>>>>>>>>>>>>>> there is interest from the SciPy team before you spend time coming up with
>>>>>>>>>>>>>> a detailed proposal. We have to determine if a project sounds feasible to
>>>>>>>>>>>>>> complete for GSOC and if appropriate mentors can be identified.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Ultimately, you will need to submit an application to SciPy
>>>>>>>>>>>>>> between March 25th - April 9th.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Cheers,
>>>>>>>>>>>>>> Greg
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>>
>>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> SciPy-Dev mailing list
>>>>>>>>> SciPy-Dev at python.org
>>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> SciPy-Dev mailing list
>>>>>>>> SciPy-Dev at python.org
>>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> SciPy-Dev mailing list
>>>>>>> SciPy-Dev at python.org
>>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>>
>>>>>> _______________________________________________
>>>>>> SciPy-Dev mailing list
>>>>>> SciPy-Dev at python.org
>>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>>
>>>>> _______________________________________________
>>>>> SciPy-Dev mailing list
>>>>> SciPy-Dev at python.org
>>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>>
>>>> _______________________________________________
>>>> SciPy-Dev mailing list
>>>> SciPy-Dev at python.org
>>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>>
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-dev
>>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at python.org
>> https://mail.python.org/mailman/listinfo/scipy-dev
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20190402/5936de07/attachment-0001.html>