python vs awk for simple sysamin tasks

[Steve Lamb]

On 2004-06-04, William Park <opengeometry at yahoo.ca> wrote:
> Too many maybe's, and you're timing hypotheoretical tasks which you hope
> it would take that long.

    No, not too many maybes.  That was based on a real-life experience which
is but a single example of many cases over the past several years of working
on my field (ISP/Web Hosting).  

> Because that's what Shell/Awk is for...  write, throw away, write, throw
> away, ...

    And you don't consider this wasteful.  Ooook then.

>>     It is amazing how much shell bogs down when you're running it over
>>     several hundred thousand directories.  :P

> Same would be true for Python here.

    Not as much as shell.  With shell you have thousands upon thousands of
fork/execs getting in the way.  In the case I am thinking of rewriting the
shell script into Perl and doing all the logic processing internally cut the
run time down from 7-8 hours down to *2*.  I have seen similar perfomance
numbers from Python when compared to shell though not on that scale.  I mean
the same could be said for C, or Assembly.  Yes, iterations over large sets of
data is going to increase runtimes.  However there are inefficiencies not
present in a scripting language which are present in shell which make it
exceptionally longer to run.

> "If hammer is all you have, then everything becomes a nail". :-)

    Fortunately I don't have just a hammer then, isn't it?  I restate, I use
shell for one liners.  Beyond that it has been my experience, practice and
recommendation that far more time is saved in the long run by using the proper
tool.  A proper scripting language and not a hobbled-together pseudo-language
with severe performance issues.

-- 
         Steve C. Lamb         | I'm your priest, I'm your shrink, I'm your
       PGP Key: 8B6E99C5       | main connection to the switchboard of souls.
-------------------------------+---------------------------------------------