Efficiency debates

[Avi Gross]

I think we have discussed this a few times. There are tradeoffs in computer science and obviously a compiled language with some optimization using low-level data structures does much better at solving simple problems.

Interpreted languages often have serious overhead to start with and allow all kinds of flexibility  that comes with more costs such as functions that handle arbitrary types of arguments.

When languages keep adding not only features but accommodate many programming styles such as object-oriented and functional or multiple ways to do the same thing, there is often more bloat. In python, there are almost always many ways to do anything!

So, I find it easier to develop in such languages BUT if the program is to be run regularly and uses lots of resources and especially if it does not need many of the dynamic features, there are ways to try to tune it within Python or use a faster attached library of functions in languages like C for some parts or leaving Python entirely. If a program does not actually need much of what Python offers, fine. Use something else. I know in another language called R, that more and more parts are rewritten in a dialect of C/C++ so some things run quite fast if they call such functions. Users learn to use vectorized methods instead of loops to gain speed.

I have recently seen discussions that some tried to solve using nontrivial regular expressions when simpler and faster methods using normal string functions would do. Unless that application also needed much more, it could easily be done in C or C++ or if needed in a very localized assembler. 

Python may be closer to one-size fits all but sometimes more than is needed. 

But it is not the only example of where people choose a sledge-hammer when a thumbtack will do. But as humans, we may do better working in ways that work well with our normal thinking process or one we can learn and feels natural. Every language I know has problems that look easy to solve using features in it and others that are painful to contemplate. Lately, I have done work combining Python parts with R parts to use the powerful features I already know how to use well in each. The new R STUDIO supports Python and quite a few other languages now as well as notebooks and document preparation and more. But I make no claim it is fast. It is very convenient and sometimes that is worth more especially for a one-off project.

Those who want speed, go for it. If you can find libraries of functions or class prototypes that are just what you need, great. But if a language like Python has even more resources available in lots of modules or packages as well as more flexible data types, then maybe a hundred lines of that will be easier to write than thousands of lines elsewhere. How convenient is it to work with data.frame type objects in C/C++ as compared to using numpy and pandas?

For many of us, we work with machines with ever more power and most of the time it just sits there idling. Now if you have something on servers that is running many simultaneous copies all the time to process complex requests from users, then efficiency should matter. If you pay for it or worry about how much electricity it uses, fine.

This forum does not care if you prefer C/C++ and is meant to discuss Python for people interested in Python. Repeated attacks make people ignore the person and not take anything they say as meaningful. I tried sharing my understanding of a recent proposal and remain unconvinced it is trivial and certainly doubt that it is doable as it is too grandiose and suggests any imaginable computer language will trivially be handled. I am sure it would have fun dealing with a programming  language in Hebrew with most text going from right to left but numbers the other way, and with added features built-in that nobody is using yet. Oh, I have no intention of doing so, but I suggest a more doable goal would be to be able to handle five procedural languages with no objects or functional programming before making bold claims. If that works, expand.

A more efficient use of my time is to tell my mailer which messages to toss.