Question about scientific calculations in Python

[Paul Hughett]

Martin Kaufmann <martinkaufmann at yahoo.com> wrote:
: On 12 Mar 2002 19:25:50 GMT, Paul Hughett
: <hughett at mercur.uphs.upenn.edu> wrote:

:>         http://www.uphs.upenn.edu/bbl/bblimage

: Thanks, I had a look at the [bblimage] webpage. It really looks interesting 
: to me. But the basic calculations really faster than the one in SciPy or
: Scientific Python?

The calculations in bblimage are probably about as fast as you can
make them without dropping into assembly and hand-tweaking for a
particular architecture.  I haven't looked much at SciPy, and don't
know how much effort they've put into performance and how the speed
compares.


:>It really depends on how many diffraction patterns you need to do.  It
:>doesn't make sense to spent two months learning C to save even 99% of
:>a ten-hour computation that you're going to do only once.  (Unless
:>your real purpose is to learn C.)  On the other hand. if you have to
:>do a few hundred patterns, then it might be a good investment of time.

: I will have to do it several times I guess... So 99% improvement in
: speed would really be nice.

I choose 99% above to make the point that even huge improvements don't
matter much for a job you're running only once.  A 10x improvement would
be more typical for a Python-to-C conversion.


: How do you glue C and Python together? Do you use any of the wrappers
: (SWIG, weave etc.) or do you do it the hard way? I just didn't
: understand the whole story about these PyObjects.

I did it the hard way, because that lets me modify to Python interface
to take advantage of Python's features and be cleaner and easier to
use than the C interface.

:>One general comment: The performance killer in many applications like
:>this is doing the innermost loop in the interpreted language (Python,
:>Matlab, etc).  If you can recast your algorithm as operations on
:>arrays rather than individual numbers, and your language does the
:>implicit loops over the elements efficiently, you typically get 10x or
:>so improvements in speed.  

: Do you mean the array calculation of NumPy?

Yes.  The loop over elements in NumPy is done in C (as far as I know)
rather than in Python, so you get that 10x speedup I mentioned above.


\begin{heresy} 

Have you considered doing this in Matlab, or its open-source clone
Octave?  Based on what little I know of your problem, it would seem
quite well suited, since it already has efficient arrays, complex
numbers, and FFTs embedded in a scripting language.

\end{heresey}


Paul Hughett