C is it always faster than nump?

[Oscar Benjamin]

On Fri, 25 Feb 2022 at 23:13, Barry <barry at barrys-emacs.org> wrote:
>
> > On 25 Feb 2022, at 23:00, Richard Damon <Richard at damon-family.org> wrote:
> >
> > On 2/25/22 2:47 PM, Chris Angelico wrote:
> >>> On Sat, 26 Feb 2022 at 05:49, Richard Damon <Richard at damon-family.org> wrote:
> >>> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> >>>> Hi,
> >>>> a lot of people think that C (or C++) is faster than python, yes I agree,
> >>>> but I think that's not the case with numpy, I believe numpy is faster than
> >>>> C, at least in some cases.
> >>>>
> >>> My understanding is that numpy is written in C, so for it to be faster
> >>> than C, you are saying that C is faster that C.
> >> Fortran actually, but ultimately, they both get compiled to machine code.
> >
> > Looking at the Github repo I see:
> >
> > Languages:
> > Python.  62.5%
> > C           35.3%
> > C++.       1.0%
> > Cython.   0.9%
> > Shell.       0.2%
> > Fortran.   0.1%
>
> I assume that this is just for bumpy and not for all its dependencies.
> That will add a lot of Fortran and c++ I expect.

NumPy links with BLAS/LAPACK that will do the heavy lifting for common
linear algebra operations. Multiple different BLAS libraries can be
used with NumPy and those libraries might be written in Fortran and
might also involve some hand-crafted assembly for particular
architectures. Some explanation of NumPy's BLAS/LAPACK support is
here:
https://numpy.org/devdocs/user/building.html

By default if you install NumPy from conda (or at least if you install
the Anaconda distribution) then I think that NumPy will use the Intel
MKL library:
https://en.wikipedia.org/wiki/Math_Kernel_Library
As I understand it the core of MKL is Fortran and is compiled with
Intel's ifortran compiler but some parts might be C/C++. MKL is also
the same library that is used by Matlab for its own heavy lifting.
It's not sufficiently free to be used in NumPy's PyPI wheels though.

If you install the precompiled NumPy wheels with pip from PyPI then I
think those are statically linked with OpenBLAS:
https://github.com/xianyi/OpenBLAS
Again I think the core of OpenBLAS is Fortran but there's some C in there.

In the pre-wheel days the situation was that NumPy provided installer
files for Windows that would give binaries linked with ATLAS (also
Fortran):
https://en.wikipedia.org/wiki/Automatically_Tuned_Linear_Algebra_Software

I think at some point NumPy used to use the OSX Accelerate library but
the page I linked above says that's now deprecated. I don't know
anything about Accelerate but I wouldn't be surprised to hear that it
was a bunch of old Fortran code!

If you build NumPy from source without having any BLAS/LAPACK
libraries then I think it uses its own backup version of these that is
written in C but not as well optimised. This used to be the default
for a pip install on Linux in pre-wheel times.

Many operations in NumPy don't actually use BLAS/LAPACK and for those
parts the heavy lifting is all done in NumPy's own C code.

Lastly SciPy which is very often used together with NumPy does have a
load of Fortran code. As I understand it at some point NumPy and SciPy
were divided from the previous numerical Python libraries and there
was a deliberate split so that the Fortran code all ended up in SciPy
rather than NumPy.

--
Oscar