[Numpy-discussion] Multidimension array access in C via Python API

[Nathaniel Smith]

On Apr 5, 2016 9:39 AM, "mpc" <matt.p.conte at gmail.com> wrote:
>
> This is the reason I'm doing this in the first place, because I made a
pure
> python version but it runs really slow for larger data sets, so I'm
> basically rewriting the same function but using the Python and Numpy C
API,
> but if you're saying it won't run any faster then maybe I'm going at it
the
> wrong way. (Why use the C function version if it's the same speed anyway?)

I haven't had a chance to look carefully at the code you posted, but the
useful general rule is that code does not magically become faster by
writing it in C -- it only becomes faster if by writing it in C you are
able to write it in a way that you couldn't from python. Those high-level
Numpy C functions you started out using from C *are* the code that you're
using from python (e.g. writing arr1 + arr2 essentially just calls
PyArray_Add and so forth); there's a little bit of pure overhead from
python itself but it's surprisingly small. (I've seen estimates of 10-20%
for regular python code, and it should be much less than that for code like
yours that's spending most of its time inside numpy.)

Where you can really win is places where you go outside the Numpy C API --
replacing an element-by-element loop with C, that sort of thing.

> You're suggesting perhaps a cython approach, or perhaps a strictly C/C++
> approach given the raw data?

Where cython shines is when you have a small amount of code that really
needs to be in C, and it's surrounded by / mixed in with code where regular
numpy operations are fine. Or just in general it's great for taking care of
all the annoying boilerplate involved in getting between python and C.

First though you need some idea of what, algorithmically, you want to
change about your implementation that will make it go faster, and why C/C++
might or might not help with implementing this strategy :-).

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20160405/5e341130/attachment.html>