[Pandas-dev] ​Re: [Numpy-discussion] Transonic Vision: unifying Python-Numpy accelerators

[PIERRE AUGIER]

Dear Pandas developers,

Ralf Gommers wrote me that there was a discussion on the pandas-dev mailing list a couple of weeks ago about adopting Numba as a dependency.

We recently wrote a serious text on this subject: http://tiny.cc/transonic-vision.

As a side remark, we also played with Transonic and Pandas in this notebook https://github.com/fluiddyn/transonic-demos/blob/master/pandas.ipynb (the binder link to run the benchmarks: https://mybinder.org/v2/gh/fluiddyn/transonic-demos/master)

> Date: Wed, 6 Nov 2019 23:49:08 -0500
> From: Ralf Gommers <ralf.gommers at gmail.com>
> To: Discussion of Numerical Python <numpy-discussion at python.org>
> Subject: Re: [Numpy-discussion] Transonic Vision: unifying
>	Python-Numpy accelerators
> Message-ID:
>	<CABL7CQhr6ps25Yrp_pAF8vWVLS3_2=Mw4bCOXb0H1dvU9AjTMw at mail.gmail.com>
> Content-Type: text/plain; charset="utf-8"
> 
> On Mon, Nov 4, 2019 at 4:54 PM PIERRE AUGIER <
> pierre.augier at univ-grenoble-alpes.fr> wrote:
> 
>> Dear Python-Numpy community,
>>
>> Transonic is a pure Python package to easily accelerate modern
>> Python-Numpy code with different accelerators (currently Cython, Pythran
>> and Numba).
>>
>> I'm trying to get some funding for this project. The related work would
>> benefit in particular to Cython, Numba, Pythran and Xtensor.
>>
>> To obtain this funding, we really need some feedback from some people
>> knowing the subject of performance with Python-Numpy code.
>>
>> That's one of the reason why we wrote this long and serious text on
>> Transonic Vision: http://tiny.cc/transonic-vision. We describe some
>> issues (perf for numerical kernels, incompatible accelerators, community
>> split between experts and simple users, ...) and possible improvements.
>>
> 
> Thanks Pierre, that's a very interesting vision paper.
> 
> In case you haven't seen it, there was a discussion on the pandas-dev
> mailing list a couple of weeks ago about adopting Numba as a dependency
> (and issues with that).
> 
> Your comment on my assessment from 1.5 years ago being a little unfair to
> Pythran may be true - not sure it was at the time, but Pythran seems to
> mature nicely.
> 
> The ability to switch between just-in-time and ahead-of-time compilation is
> nice. One thing I noticed is that this actual switching is not completely
> fluent: the jit and boost decorators have different signatures, and there's
> no way to globally switch behavior (say with an env var, as for backend
> selection).
> 
> 
>> Help would be very much appreciated.
>>
> 
> I'd be interested to help think about adoption and/or funding.
> 
> Cheers,
> Ralf
> 
> 
>>
>> Now a coding riddle:
>>
>> import numpy as np
>> from transonic import jit
>>
>> @jit(native=True, xsimd=True)
>> def fxfy(ft, fn, theta):
>>     sin_theta = np.sin(theta)
>>     cos_theta = np.cos(theta)
>>     fx = cos_theta * ft - sin_theta * fn
>>     fy = sin_theta * ft + cos_theta * fn
>>     return fx, fy
>>
>> @jit(native=True, xsimd=True)
>> def fxfy_loops(ft, fn, theta):
>>     n0 = theta.size
>>     fx = np.empty_like(ft)
>>     fy = np.empty_like(fn)
>>     for index in range(n0):
>>         sin_theta = np.sin(theta[index])
>>         cos_theta = np.cos(theta[index])
>>         fx[index] = cos_theta * ft[index] - sin_theta * fn[index]
>>         fy[index] = sin_theta * ft[index] + cos_theta * fn[index]
>>     return fx, fy
>>
>> How can be compared the performances of these functions with pure Numpy,
>> Numba and Pythran ?
>>
>> You can find out the answer in our note http://tiny.cc/transonic-vision
>> :-)
>>
>> Pierre
>>
>> > Message: 1
>> > Date: Thu, 31 Oct 2019 21:16:06 +0100 (CET)
>> > From: PIERRE AUGIER <pierre.augier at univ-grenoble-alpes.fr>
>> > To: numpy-discussion at python.org
>> > Subject: [Numpy-discussion] Transonic Vision: unifying Python-Numpy
>> >       accelerators
>> > Message-ID:
>> >       <
>> 1080118635.5930814.1572552966711.JavaMail.zimbra at univ-grenoble-alpes.fr>
>> >
>> > Content-Type: text/plain; charset=utf-8
>> >
>> > Dear Python-Numpy community,
>> >
>> > Few years ago I started to use a lot Python and Numpy for science. I'd
>> like to
>> > thanks all people who contribute to this fantastic community.
>> >
>> > I used a lot Cython, Pythran and Numba and for the FluidDyn project, we
>> created
>> > Transonic, a pure Python package to easily accelerate modern
>> Python-Numpy code
>> > with different accelerators. We wrote a long and serious text to explain
>> why we
>> > think Transonic could have a positive impact on the scientific Python
>> > ecosystem.
>> >
>> > Here it is: http://tiny.cc/transonic-vision
>> >
>> > Feedback and discussions would be greatly appreciated!
>> >
>> > Pierre
>> >
>> > --
>> > Pierre Augier - CR CNRS                 http://www.legi.grenoble-inp.fr
>> > LEGI (UMR 5519) Laboratoire des Ecoulements Geophysiques et Industriels
>> > BP53, 38041 Grenoble Cedex, France                tel:+33.4.56.52.86.16
>> _______________________________________________