C is it always faster than nump?

[Avi Gross]

I agree with Richard.

Some people may be confused and think c is the speed of light and relativistically speaking, nothing can be faster. (OK, just joking. The uses of the same letter of the alphabet are not at all related. One is named for the language that came after the one named B, while the other may be short for celeritas meaning speed.)

There is no such thing as C. C does nothing. It is a combination of a language specification and some pieces of software called compilers that implement it well or less well.

There is such a thing as a PROGRAM. A program completely written in C is a thing. It can run fast or slow based on a combination of how it was written and on what data it operates on, which hardware and OS and so on. AND some of it may likely be running code from libraries written in other languages like FORTRAN that get linked into it in some way at compile time or runtime, and hooks into the local OS and so on.

So your program written supposedly in pure C, may run faster or slower. If you program a "sort" algorithm in C, it may matter if it is an implementation of a merge sort or at bubble sort or ...

As noted, numpy is largely written in C. It may well be optimized in some places but there are constraints that may well make it hard to optimize compared to some other implementation without those constraints. In particular, it interfaces with standard Python data structures at times such as when initializing from a Python List, or List of Lists, or needing to hold on to various attributes so it can be converted back, or things I am not even aware of.

So, I suspect it may well be possible to make a pure C library similar to numpy in many ways but that can only be used within a C program that only uses native C data structures. It also is possible to write such a program that is horribly slow. And it is possible to write a less complex version of numpy that does not support some current numpy functionality and overall runs much faster on what it does support.

I do wonder at the reason numpy and pandas and lots of other modules have to exist. Other languages like R made design choices that built in ideas of vectorization from the start. Python has lots of object-oriented extensibility that can allow you to create interpreted code that may easily extend it in areas to have some similar features. You can create an array-like data structure that holds only one object type and is extended so adding two together (or multiplying) ends up doing it componentwise. But attempts to do some such things often run into problems as they tend to be slow. So numpy was not written in python, mostly, albeit it could have been even more impressive if it took advantage of more pythonic abilities, at a cost.

But now that numpy is in C, pretty much, it is somewhat locked in when and if other things in Python change. 

The reality is that many paradigms carried too far end up falling short.


-----Original Message-----
From: Richard Damon <Richard at Damon-Family.org>
To: python-list at python.org
Sent: Fri, Feb 25, 2022 1:48 pm
Subject: Re: C is it always faster than nump?


On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:
> Hi,
> a lot of people think that C (or C++) is faster than python, yes I agree,
> but I think that's not the case with numpy, I believe numpy is faster than
> C, at least in some cases.
>
My understanding is that numpy is written in C, so for it to be faster 
than C, you are saying that C is faster that C.

The key point is that numpy was written by skilled programmers who 
carefully optimized their code to be as fast as possible for the major 
cases. Thus it is quite possible for the numpy code to be faster in C 
than code written by a person without that level of care and effort.

There are similar package available for many languages, including C/C++ 
to let mere mortals get efficient numerical processing.

-- 
Richard Damon


-- 
https://mail.python.org/mailman/listinfo/python-list