scripting languages vs statically compiled ones

[kosh]

On Thursday 28 October 2004 3:08 pm, Cameron Laird wrote:

> I want to reinforce one of Kosh's observations:  for real-world
> applications, Python is often FASTER than C.  Not at least a
> hundredth as fast, or at least a tenth as fast, but the-customers-
> complain-about-the-C-coded-version-but-love-the-Python-based-one:
> FASTER.
>

Here is the method I see on making programs fast in python.

Step 1: Make sure the program works  (Who cares how slow it is if it does not 
get the job done)

Step 2: Go through the program and see where you can simplify it. I know many 
find it amazing but overall the shorter and simpler your python code is the 
faster it tends to run. However the point of this step is not speed that is 
just a side benefit. The point is that your code is now easier to read, fix 
etc.

Step 3: If during the writing of your code and cleaning it up you are not 
duplicating features available in the standard python install use the built 
in ones instead. They get a lot more testing, are far less likely to contain 
bugs, it is code you don't have to maintain etc. This will make your code 
even simpler to read and understand and also shorter and will likely get rid 
of more bugs.

Step 4: Profile your code. Now that your code is nice and clean it is far 
easier to understand. This also makes the profiler information far more 
useful since you can more easily see what the problem is. At this point think 
carefully about the design and algorithm choice. Fix those problems first and 
then fix up the speed issues by using better designs in python. Make sure 
while you are fixing up speed issues that the code is still short and simple.

Step 5: If you are on a system that psyco works for use that sometimes it 
helps a lot and it trivial to add.

Step 6: Look for outside libraries to use like numarray, pil etc. It is better 
to reuse code then it is to write more especially if you try and stay with 
reusing larger projects since they get a lot more work on optimization and 
debugging then you can do. Besides reinventing the wheel takes too much time.

By this point your code should be pretty darn fast, short, easy to understand 
and fairly easy to debug.

Step 7: As a last resort if your program is still not fast enough go ahead and 
write code in a lower level language and call it from python. However be 
careful with this step and make sure you really need to do it. The code you 
write in a lower level language is far more likely to be buggy then anything 
else up to this point and it will also usually take a fair bit of time to 
actually write something in a low level language faster then what the above 
steps have aleady given in gains.

At any point along this process you can stop if your code is fast enough and 
for most people there is no need to go beyond step 6. 

Remember the point of all of this is to get good software written not brag 
about how cool you are for writing stuff in a lower level language and 
wasting a customers time and money. Doing as much as possible in python and 
using libraries saves a lot of time and money and gives a better result 
overall. Then end result of this is also your programs tend to be a lot 
faster then their low level language counterparts since it takes so much time 
to optimize in a low level language. So while others still have a program 
that will often times before 2-10x slower despite being in a "faster 
language" you are working on adding new features, makng your app even faster 
etc.


Another minor point for this is that each new python version often adds a lot 
of very useful features for making your code simpler, faster, more memory 
efficient etc. It is often worth it to take the time and convert the program 
to use the features in a newer version of python. Things like sets, 
generators, genexps etc can save a lot of code and memory.