[SciPy-user] algorithm, optimization, or other problem?

[Andrew Straw]

Brian Blais wrote:

>Given the huge difference between the Pyrex and the Mex, I feel that there is 
>something I am doing wrong, because the C-code for both should run comparably. 
>Perhaps the approach is wrong?  I'm willing to take any suggestions!  I don't mind 
>coding some in C, but the Python API seemed a bit challenging to me.
>
>  
>

Dear Brian,

The answer is that most of your code is still using the (slow) Python
API. Take a look at the c files that pyrex produces, particularly for
your time critical loop. I suspect you'll be horrified to see how much
stuff like the following there is.

      __pyx_3 = PyObject_CallObject(__pyx_12, __pyx_1); if (!__pyx_3)
{__pyx_fi\lename = __pyx_f[0]; __pyx_lineno = 469; goto __pyx_L1;}

These are calling the Python interpreter, which is quite expensive, as
you've noticed. Fortunately, you'll be able to get rid of 99% of this
stuff by clever use of Pyrex. Still, I'm actually fairly impressed that
you got as much of a speedup as you did.

What you need to do is convert  your code to raw C array-element access
to bypass the Python interpreter as much as possible. See the
http://scipy.org/Wiki/Cookbook/Pyrex_and_NumPy demo for an example,
particularly the print_elements() function. You can reduce the
complexity of that code considerably if you deal only with contiguous,
1D, single dtype arrays.

Finally, it's possible that implementing this in weave or some other
wrapping solution would more straightforward. I prefer Pyrex for its
general Python/C binding properties, not necessarily for any potential
ease-of-array-manipulation. (Plus it has a syntax I like! :) For that
matter, you could implement your function in pure C and use Pyrex (or
whatever) simply to interface it.