Idiom for running compiled python scripts?

[Steven D'Aprano]

On Fri, 23 Mar 2007 12:22:44 +0000, mark wrote:

> On Fri, 23 Mar 2007 22:24:07 +1100, Steven D'Aprano wrote:
>>     if not os.path.exists(compiledname) or \ os.stat(compiledname)[MT] <
>>     os.stat(scriptname)[MT]:
>>         # compiled file doesn't exist, or is too old
> 
> Surely the validity check done by Python is more sophisticated than
> this? Doesn't the binary file have to be compiled under the same python
> version etc?

Of course. What, do you want me to do all your work? :-)



>> Now don't forget to test whether launching the subshell takes longer
>> than the 20ms you might save. All that effort, and wouldn't it be ironic
>> if it was actually *slower* than executing the script from scratch each
>> time...
> 
> But Python proper is executing all the above anyhow isn't it? So the 20
> msecs advantage I measured already includes this logic.

I don't know how you measured the 20ms.

When you call a script direct from the shell, you've already got a shell
running. When you call a script from within Python via os.system, it has
to launch a sub-shell. That takes time.


> Anyhow, I give up. Compilation, it seems, only applies to python
> modules. Compilation is not appropriate for Python scripts. Should be
> in the FAQ.

No, that's not true. Python scripts certainly take advantage of compiled
modules.

The real lesson of this is that optimization isn't necessarily
straightforward. What we imagine "should be" faster might not be in
practice -- especially when dealing with micro-optimizations that only
save a few tens of milliseconds.

Frankly, it simply isn't worth trying to save 20ms in a script that takes
less than a second to run. If you scratch your nose before hitting enter,
you've wasted a hundred times what you've just spent hours trying to save.

Or, to put it another way:

The Rules of Optimization are simple. 
Rule 1: Don't do it. 
Rule 2 (for experts only): Don't do it yet.
-- Michael A. Jackson (no, not that Michael Jackson), "Principles of
Program Design", 1975.



-- 
Steven.