threading

[Sturla Molden]

Paul Rubin <no.email at nospam.invalid> wrote:
> Sturla Molden <sturla.molden at gmail.com> writes:
>> When should we use C++ or Fortran instead of Python? Ever?
> 
> When performance matters? 

Sometimes, but usually performance only matters in certain types of
calculations like matrix algebra or FFTs. But we always use specialized
libraries for that, such as Intel MKL, OpenBLAS, AMD ACML, or Apple's
Accelerate Framework.  "When performance matters" is not really a good
answer, because performance might not matter where we think or as much as
we think.

For examle if I were to write a Levenberg-Marquardt solver, I would e.g.
find that 95% of the time is spent solving the linearized least-squares
equations

   (J' J + lambda diag(J' J)) dx = J' error

using QR or SVD. If I used LAPACK method DGELS or DGELSS to solve these
equations with QR or SVD, I would find that my otherwise idential Fortran
and Python code would perform roughly the same. None of the performance
libraries I mentioned cares if I am calling them from Python or Fortran.
The differences in runtime would thus be in the 5 % spent outside the
performance libraries. And if i/o is more efficient in Python, which is
very likely, it might even be that the Python version runs faster. So the
question is, when does it matter for me? Now and then, perhaps, but very
rarely. For example it did matter when we wrote the cKDTree object for
SciPy. But it never matters when I use it. 

I will make the bold claim that 90 % of the use of C++ or Fortran code in
scientific computing stems from the misconception that "writing everything"
in a compiled language will be faster. I have done this mistake myself, and
lost valuable time. I have promised myself I will not do it again, and then
realized I have done the same mistake yet again. It is a very dangerous
misconception. Always prototype in Python first, then profile, then choose
between optimization or using a bigger computer. Starting out by writing C,
C++, Cython or Fortran is a trap. Yet many do, myself included, because we
errorneously trust ourselves to know when we should just use Python. But
the thing is, nobody does, we just think we do. 

Sturla