[Tutor] improving speed using and recalling C functions

[Martin A. Brown]

Hi there Gabriele,

 : I have a program that is reading near 600000 elements from a 
 : file. For each element it performs 200 times a particular 
 : mathematical operation (a numerical interpolation of a function). 
 : Now these process takes near 8 hours.

Sounds fun!  Here are some thoughts (though I may not have any solid 
answers, I hope these pointers are useful):

  Are you sure (from profiling) that the numerical interpolation
    is the bottleneck?

  What is the numerical operation?
    1: Is the function implemented in Numpy? [0]
    2: How about SciPy? [1]

Is there a domain-specific Python library that already does the 
computation you want to achieve?  If so, then you have "only" the 
format question--how to put your data into the format required by 
the library.  If not, then on to the next question you ask ...

 : Creating a C function and calling it from the code could improve 
 : the speed? It could be useful that it contains both the 
 : mathematical operation and the for loop or only the mathematical 
 : operation?
 : 
 : If it is a reasonable solution to try decreasing the 
 : computational time how can I implement this C function?

It is certainly an option!  In the event that you cannot find 
something that is fast enough in Python already and you cannot find 
any C library that has Python bindings, then, whether you have to 
write your own, or you have a third party C library, you have 
several options for writing bindings.

You can write your Python bindings using:

  * ctypes: https://docs.python.org/2/library/ctypes.html
  * cffi: http://cffi.readthedocs.org/en/release-0.8/
  * cython: http://www.cython.org/
  * roll your own C!

It might be beyond the scope of this list (certainly beyond my 
capabilities) to provide recommendations on which avenue to 
take--but if you are asking, you may already have the tools you need 
to assess for yourself.  There is, however, quite a bit of 
experience loitering around this list, so perhaps somebody else will 
have a suggestion.

-Martin

 [0] http://www.numpy.org/
 [1] http://www.scipy.org/

-- 
Martin A. Brown
http://linux-ip.net/