[SciPy-Dev] Backend support for fftpack - GSoC 2019
Hameer Abbasi
einstein.edison at gmail.com
Mon Jun 10 16:22:48 EDT 2019
On Mon, 2019-06-10 at 19:51 +0000, Peter Bell wrote:
> >This was based on the feedback of the NumPy team
>
>
>
> Is this feedback available anywhere? I’ve had a look through the
> NumPy-Discussions list but couldn’t see anything.
Unfortunately not, this was at the NumPy dev summit at Berkeley and
done in person.
>
>
> > If you have code that is dependent on the last version, I’d be
> happy to revert my changes.
>
>
>
> Don’t worry, I just want a sense of where the project is currently.
That's great. Maybe I should be clearer in my wording: NumPy and SciPy
are major potential users, and I'm willing to take uarray in the
direction that's needed for those projects.
>
> Peter
>
>
>
> From: SciPy-Dev <scipy-dev-bounces+peterbell10=live.co.uk at python.org>
> On Behalf Of Hameer Abbasi
>
> Sent: 10 June 2019 19:40
>
> To: SciPy Developers List <scipy-dev at python.org>
>
> Subject: Re: [SciPy-Dev] Backend support for fftpack - GSoC 2019
>
>
>
>
>
> Hi Peter,
>
>
>
>
>
> This was based on the feedback of the NumPy team, which preferred
> protocols over an OOP based model and wanted to avoid deep nesting.
> It also bought down the overhead factor tenfold.
>
>
>
>
>
> I also removed an arg/kwarg normalisation feature, which bought down
> the overhead another 15-fold.
>
>
>
>
>
> Ralf and I had some offline discussions about the overhead, which
> seemed to be more of a blocker than all of the other features
> combined.
>
>
>
>
>
> Rest assured, this was out of a desire to respond to feedback from
> the NumPy and SciPy teams and once there is code actually relying on
> uarray, we will not change the API any further.
>
>
>
>
>
> If you have code that is dependent on the last version, I’d be happy
> to revert my changes.
>
>
>
>
>
>
>
>
> Best Regards,
>
> Hameer Abbasi
>
>
>
>
>
>
>
> >
> > On Monday, Jun 10, 2019 at 4:49 PM, Peter Bell <
> > PeterBell10 at live.co.uk> wrote:
> >
> >
> >
> > I’ve had another look at the uarray docs and was quite surprised to
> > see that the API has changed significantly since I last looked at
> > them.
> > It seems that uarray is now based on a __ua_function__
> > protocol which is similar to NEP 18 except that it’s held in the
> > backend rather than the array type. I think I’ve tracked the
> > change to
> > PR #152 which is dated after the API was claimed to be stable.
> >
> >
> > Hameer, would you be able to clarify what state uarray development
> > is in: exactly how stable is the API?
> >
> > Peter
> >
> >
> >
> >
> > From: SciPy-Dev <
> > scipy-dev-bounces+peterbell10=live.co.uk at python.org>
> > On Behalf Of Hameer Abbasi
> >
> > Sent: 09 May 2019 13:39
> >
> > To: SciPy Developers List <scipy-dev at python.org>
> >
> > Subject: Re: [SciPy-Dev] Backend support for fftpack - GSoC 2019
> >
> >
> >
> >
> >
> >
> >
> > Hi,
> >
> >
> >
> >
> >
> > Since uarray came up: I’m the author. Yes, performance is slow
> > (since it’s pure python) but the API is stable at this point (the
> > docs are a bit out of
> > date, I should update them).
> >
> >
> >
> >
> >
> > We’re welcoming any kind of contributions as I, myself, am not
> > familiar enough with the CPython API to do it. However, I’d love to
> > learn!
> >
> >
> >
> >
> >
> > It does have a focus on array computing but it is meant to be
> > fairly general, and you shouldn’t have any problems with the
> > interface.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > Best Regards,
> >
> > Hameer Abbasi
> >
> >
> >
> >
> >
> >
> >
> > >
> > > On Thursday, May 09, 2019 at 8:25 AM, Peter Bell <
> > > PeterBell10 at live.co.uk> wrote:
> > >
> > >
> > >
> > > >> Should backends be allowed to add functionality or should they
> > > be completely
> > > >> interchangeable. E.g. unlike fftpack, FFTW has support for
> > > long doubles;
> > > >> should this be exposed to SciPy users?
> > > >
> > > > If a user writes backend-specific code, then they may just as
> > > well bypass the
> > > > backend right, and call the other library directly?
> > >
> > > I agree, in fact I quote you in the proposal saying something
> > > similar.
> > >
> > > > Although for long double things will work out of the box
> > > perhaps, because the
> > > > dtype of an input array doesn't come back in the API. So the
> > > array can be
> > > > passed straight to the backend.
> > >
> > > This was why I picked it as an example. It doesn't add any new
> > > parameters or
> > > change the function signature at all, but it does change the
> > > result of a
> > > particular call to the API. So it's a very minimal change yet it
> > > still breaks
> > > the interchangeability of backends.
> > > > This is probably also a good time to point out
> > > http://uarray.readthedocs.io/.
> > > > It can provide a complete backend system without too much work
> > >
> > > After reading through the docs it does seem to cover this and
> > > more. However, it
> > > also seems to be unstable at this point so I’m not sure it’s wise
> > > to rely on it just
> > > yet.
> > >
> > > > it was designed exactly for the type of problem you're trying
> > > to solve here.
> > >
> > > Unless I'm misunderstanding it seems to be more focused on
> > > supporting different
> > > types of array, rather than different implementations for NumPy
> > > arrays. I can't
> > > see any utility in having a complicated dispatch mechanism when
> > > we really only
> > > want to have a single backend at a time for any given function.
> > >
> > > I would assume that because of this, uarray would come with a
> > > greater
> > > performance overhead. Whereas I think the overhead of an fftpack
> > > backend should
> > > just be some argument validation and a single function call. I
> > > could always
> > > implement it both ways and benchmark it if there's interest.
> > >
> > > Peter
> > >
> > > From: SciPy-Dev <
> > > scipy-dev-bounces+peterbell10=live.co.uk at python.org>
> > > On Behalf Of Ralf Gommers
> > >
> > > Sent: 08 May 2019 23:59
> > >
> > > To: SciPy Developers List <scipy-dev at python.org>
> > >
> > > Subject: Re: [SciPy-Dev] Backend support for fftpack - GSoC 2019
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > On Wed, May 8, 2019 at 5:50 PM Peter Bell <PeterBell10 at live.co.uk
> > > > wrote:
> > >
> > > >
> > > > Hello all,
> > > >
> > > > I’m happy to say my GSoC proposal was accepted,
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > Hey Peter, congratulations! We're happy to have you on board!
> > >
> > >
> > >
> > >
> > > >
> > > > so I’ll be working over the summer to add support for multiple
> > > > fftpack backends. For those interested, I’ve extracted the main
> > > > discussion from my proposal which you can read here:
> > > >
> > > > https://docs.google.com/document/d/1hGMc6ZrsmSi_LSixI5oFbo5cZkPUo92sfmnVthg1u6Y/edit
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > Thanks, interesting. Will take a little while to process. I just
> > > give some very initial thoughts below (so take with a grain of
> > > salt).
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > > There are a several design decisions I raise in the proposal
> > > > and any comments would be appreciated. Of particular note:
> > > >
> > > > * Should backends be allowed to add functionality or should
> > > > they be completely interchangeable. E.g. unlike fftpack, FFTW
> > > > has support for long doubles; should this be exposed to SciPy
> > > > users?
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > If a user writes backend-specific code, then they may just as
> > > well bypass the backend right, and call the other library
> > > directly? Although for long double things will work out of
> > > the box perhaps, because the dtype of an input array doesn't
> > > come back in the API. So the array can be passed straight to the
> > > backend.
> > >
> > >
> > >
> > >
> > > >
> > > > * Is there any interest in adding a SciPy config file? For just
> > > > one option it’s probably not worthwhile but if there is
> > > > interest in more config options then it starts to make more
> > > > sense.
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > My sense is no. Something like a context manager (`with
> > > backend('pyfftw'):`) is lightweight enough probably.
> > >
> > >
> > >
> > >
> > >
> > > This is probably also a good time to point out
> > > http://uarray.readthedocs.io/. It can provide a complete backend
> > > system without too much work, it was designed exactly for the
> > > type of problem you're trying to solve here. Would be good to
> > > have your thoughts on that.
> > >
> > >
> > >
> > >
> > > >
> > > >
> > > > Ideally a clear set of design goals can be agreed upon now,
> > > > before it gets too far into the coding period which begins at
> > > > the end of the month.
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > >
> > > Yes great idea.
> > >
> > >
> > >
> > >
> > >
> > > Cheers,
> > >
> > >
> > > Ralf
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > _______________________________________________
> > >
> > >
> > > SciPy-Dev mailing list
> > >
> > > SciPy-Dev at python.org
> > >
> > > https://mail.python.org/mailman/listinfo/scipy-dev
> > >
> > >
> > >
> >
> > _______________________________________________
> >
> > SciPy-Dev mailing list
> >
> > SciPy-Dev at python.org
> >
> > https://mail.python.org/mailman/listinfo/scipy-dev
> >
> >
> >
>
>
>
>
> _______________________________________________SciPy-Dev mailing
> listSciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20190610/1deaa857/attachment-0001.html>
More information about the SciPy-Dev
mailing list