C is it always faster than nump?

[Avi Gross]

Is that fair, Grant?

I go back far enough that in my earliest years I was submitting FORTRAN programs written on punched cards and often getting them back the next day. The compiling was not the major factor in how long it took.

For many cases, a compiled language only needs to be compiled once and can be run many times. Other than during development timeframes, the major concept of speed is how well it runs not how long it takes to compile, especially if the compiler is busy doing all kinds of optimizations to your code or checking for possible obscure errors or making sure the code does not try something funny like access memory not allowed and so on.

An interpreted language often has to do the same things every time, albeit some have come up with ways to partially compile modules and only redo if they have been changed. Some of what they do every time is in some sense wasted effort but as a tradeoff, they can do many things in a dynamic way that compiled programs may not do easily or at all. 

Another argument I find is unfair, is some comparisons with what I consider "mixed" effort. If you have a program that calls various numpy routines using native python in between, then clearly a decent amount of the time spent is not in numpy. It may for example suddenly import a module and that takes time spent doing nothing about the problem other than loading what is needed. Another example, it is common to place the numbers or other values you get from numpy operations into dictionaries, lists and so on, or to make graphs. If you had done the same work in a purely C (or FORTRAN or whatever environment) and had access to similar other functionality, the latter two would all be in C or some compiled library. 

With exceptions aplenty, speed is not always a primary consideration. Often programmer time and productivity matter more. I have seen many projects though that were first implemented in a language like Python and when they had figured out a decent way to do the task reliably, they handed it over to people to redo much or all of it using languages like C++, compiles and all, as a reasonable translation from a working application may take much less time to implement and once done, way work better. At least it may until you want to change it! Prototyping often works better in some languages. Python has a compromise that modules can use to speed up important parts of the process by substituting a compiled function in C/C++/whatever for a local Python function but not necessarily switching entirely to C and suffering under the negatives of that environment.

I do wonder if we are already on a path where a language that handles concepts like parallelism (or even weirder, quantum computations) well may be a better candidate for doing some projects well as even though it may not run at top speed, it can make use of many processors or a network of machines in the cloud, to handle things in a more flexible way that may even get the job done faster.

Next discussion is whether pandas is faster than C, followed by SciPy ...

I do have to wonder if anyone ever considered adding back enough functionality into base Python to make some additions less needed. Is there any reason the kind of structures used by so many languages cannot be made part of python such as a vector/array that holds exactly one kind of data structure and not force use of things like a list when that is more than is needed?


-----Original Message-----
From: Grant Edwards <grant.b.edwards at gmail.com>
To: python-list at python.org
Sent: Fri, Feb 25, 2022 4:12 pm
Subject: Re: C is it always faster than nump?


On 2022-02-25, Richard Damon <Richard at Damon-Family.org> wrote:

> On 2/25/22 4:12 AM, BELAHCENE Abdelkader wrote:

>> Hi,

>> a lot of people think that C (or C++) is faster than python, yes I agree,

>> but I think that's not the case with numpy, I believe numpy is faster than

>> C, at least in some cases.

>

> My understanding is that numpy is written in C,



Sort of. IIRC a lot of the heavly lifting is done by libraries like

BLAS and LAPAK that were written in FORTRAN the last time I checked.



Has that changed?



Or am I conflating numpy with some other scientific-python stuff. Back

when I did a lot of numerical stuff in Python, I remember spending a

lot of time watching FORTRAN compiles. Admittedly, that's getting to

be 10+ years ago...



--

Grant





-- 

https://mail.python.org/mailman/listinfo/python-list