Python as a numerical prototyping language.

[Johannes.Nix at web.de]

Carl <phleum_nospam at chello.se> writes:

> Now, I have always believed that Python is a poor performer in terms of
> numerical speed. My experience, however, is that the efficient use of
> dictionaries, lists, precompiled numerical libraries, etc makes Python a
> higly competitive language for scientific computing (in comparison with
> more traditional languages such as C, C++ and different versions of
> Fortran).

First, look at http://www.python.org/topics/scicomp/ .

I am using Python + Numeric since a few years for really
computing-intensive signal processing and real-time audio processing
(explorative noise reduction for hearing aids). I code at least twenty
times faster than I did in pure C.  It works great and on some
occations execution speed has been five times faster than Matlab; on a
Linux system with kernel modifications, Python with C extensios can
achieve worst-case latencies of 20 milliseconds or less for
full-duplex audio processing.  At the other hand side, I use it for
really computing-intensive stuff on a Linux cluster (LAM) with 32
nodes. Plus, it runs neatly on multiprocessor Alpha or Opteron
systems, and can run in embedded systems; no need for a graphic card.

Another quality of Numerical Python that it is very easy to integrate
C code and very mature stuff from Netlib. Scientfic code can be very
difficult to test and verify so a good code base for numerical
routines is paramount. An interesting project for Python would be to
integrate the GNU scientific library and important Netlib routines as
far as possible. The module philosophy of Python, as well as Python's
community culture, supports it very well to use existing code again.

One thing that is quite easy to do is to start coding everything in
Python and develop the algorithm; then identify the hot spots by using
the profiler, and move them to C. Something of the complexity of a MP3
player with net access and graphic user interface can be easily
implemented in Python+C today.

One point about Matlab: The basic Matlab is _expensive_. But in a
commercial environment Matlab with signal processing extensions costs
_tens of thousands of dollars_.  If I were founding an engineering
startup, I would avoid Matlab if possible at all.

> 
> Where is Python heading? Will it be a viable replacement for traditional
> number-crunching languages in the near-term future?
> 

For applications like audio, script languages will replace C and C++
widely in few years, because processors are just fast enough.

In my eyes, what Python, and Open Source in general, is still lacking
is an very easy, fast and very high-quality library for scientific and
financial plotting, although there are lots of good and thougthful
beginnings like Gnuplot, OpenDX, and many more. This would be an
essential building block to foster the use of Python in science.

Another thing that is a hassle for real-time programming on
multiprocessor systems is the Global Interpreter Lock (GIL). For
signal processing and real time replacing the GIL by per-object locks
would be an advance comparable to Linux-2.0. Currently the best
workaround seems to be the use of shared memory and LAM (by the pypar
module).

Johannes

P.S.

The mail address above traps spam. Mail to 
string.replace(address, 'web', 'uni-oldenburg')