[SciPy-User] Should one NOT start with Cython first?

[George Nurser]

I've found numba very good, and the mailing list helpful.
It's new, and under pretty active development, so I guess it's not
surprising that it's presently less used than cython. But there's e.g.
Stephan Hoyer's numbagg
https://github.com/shoyer/numbagg

To build it yourself may be a pain, but, it is very easy to install it with
conda install numba
if you have the anaconda distribution, and, on the mac, with macports,
sudo port install py34-numba (or py27-numba)
works like a charm. Both macports and conda give up to date versions.

cheers, George Nurser.

On 15 March 2015 at 00:24, Matthew Brett <matthew.brett at gmail.com> wrote:

> Hi,
>
> On Fri, Mar 13, 2015 at 10:25 PM, Scott High <highscot at gmail.com> wrote:
> > Hey All,
> >
> > First:
> >>
> >> I have promised myself to never start writing "Cython-first" in mind
> >> again.
> >
> > I agree, good choice.
> >
> > Beyond that, I have a few comments based on my experience using
> > Python/Cython for research in numerical methods.
> >
> >> - Most code in scientific programming is not CPU bound. You don't get a
> >> faster harddrive or faster network connection from using Cython. This
> >> accounts for the majority (often 80 to 90 %) of the code we write.
> >
> > This is often true, but is not necessarily an argument again using
> Cython.
> > (You only mentioned hard drives and network connections, but hitting any
> > level in the memory hierarchy that is not cache can wreck performance).
> If
> > you carefully read code that uses numpy array operations (especially
> > slicing) you will notice that data will often be accessed multiple times
> in
> > the same mathematical operation. The largest speed ups I have gotten with
> > Cython are not from reducing the number of flops, but in using C style
> loops
> > to minimize the number of passes through memory. For the computer
> scientists
> > in the room: Cython can often allow you to better utilize cache. For me a
> > good rule has been: usually don't convert to Cython to reduce the number
> of
> > flops, but consider converting to better manage memory. There are some
> > serious weasel words in that rule, mostly because it is very hard to
> judge
> > potential speed ups by inspecting code, even for experts.
> >
> >> They key here is that Cython is really great, and can be very easy to
> use
> >> -- but it is not at all cost free. The edit-compile-test cycle is much
> >> slower, and it's a lot harder to debug than Python -- so you really
> don't
> >> want to use it unless there is a real gain to to doing so.
> >
> > If you intend to write the majority of your program in Cython the compile
> > time can really hurt you, and I am not crazy about the large amount of
> > unreadable code generated. On the other hand, I have found that if you
> only
> > intend to convert a small number of functions then the IPython Cython
> cell
> > magic can make development nearly as quick as it would be with pure
> Python.
> > As far as debugging goes, it is a pain. No way around that one.
> >
> >> - Optimizing NumPy code with Numba is often easier than using Cython.
> >
> > I have used Numba and was very much impressed. The problem is that
> > installation can be a nightmare. Depending on your users system they may
> > have to install LLVM (I had to build it from source). This is a very
> heavy
> > duty dependency, do not discount it.
>
> Yes, I would think hard before committing to numba.  Maybe because of
> dependencies, or some other reason, I believe it is fair to say that
> Cython is in much wider use than numba.  For example, I don't know if
> any use of numba in the packages I use (or write) but Cython is very
> common.
>
> Cheers,
>
> Matthew
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20150315/8c07f4b9/attachment.html>