Python extension performance

[Jack Diederich]

On Fri, Apr 08, 2005 at 10:14:52PM -0400, David Jones wrote:
> I am trying to hunt down the difference in performance between some raw 
> C++ code and calling the C++ code from Python.  My goal is to use Python 
> to control a bunch of number crunching code, and I need to show that 
> this will not incur a (big) performance hit.
..
> My C++ function (testfunction) runs in 2.9 seconds when called from a 
> C++ program, but runs in 4.3 seconds when called from Python. 
> testfunction calculates its own running time with calls to clock(), and 
> this is for only one iteration, so none of the time is in the SWIG code 
> or Python.
..
> Some potential causes of my problems:
> 
> - linking to a shared library instead building a static exe.
> - intel libraries are not being used when I think they are
> - libpython.so was built with gcc, so I am getting some link issues
> - can linking to python affect my memory allocation and deallocation in 
> c++??
The main overhead of calling C/C++ from python is the function call overhead
(python creating the stack frame for the call, and then changing the python
objects into regular ints, char *, etc).  You don't mention how many times
you are calling the function.  If it is only once and the difference is 1.4
seconds then something is really, really, messed up.  So I'll guess it is
hundreds of thousands of times?  Let us know.

> Some things I can try:
> - recompile python with the intel compiler and try again
> - compile my extension into a python interpreter, statically
> - segregate the memory allocations from the numerical work and compare 
> how the C++ and Python versions compare
Recompiling with the Intel compiler might help, I hear it is faster than 
GCC for all modern x86 platforms.  I think CPython is only tested on GCC
and windows Visual-C-thingy so you might be SOL.  The other two ideas
seem much harder to do and less likely to show an improvement.

-jackdied