From Perl to Python: restructuring a HPC workflow

[rusi]

On Mar 27, 5:58 pm, Chris Angelico <ros... at gmail.com> wrote:
> On Wed, Mar 27, 2013 at 10:29 PM, neurino <lelli.l... at googlemail.com> wrote:
> > We are a small group of people (approx. 10), working separetely on their own
> > projects (each employee manages approx. 2-3 projects). We deal with high
> > loads of data everyday.
>
> > This workflow has been flawless now for at least 15 years. New generations
> > of employees have been given Perl scripts and they developed the tools
> > further.
>
> I would recommend making sure the tools can all interoperate
> regardless of language, and then you can change any one at any time.
> Chances are that's already the case - working with stdin/stdout is one
> of the easiest ways to do that, for instance. With a structure that
> lets anyone use any language, you can then switch some of your things
> to Python, and demonstrate the readability advantages (which would you
> rather code in, pseudocode or line noise?). Make the switch as smooth
> as possible, and people will take it when it feels right.
>
> ChrisA

What Chris says is fine in the technical sphere.
It seems to me however that your problems are as much human as
technical -- convincing entrenched old fogeys to change.
No I dont have any cooked answers for that… You just need to keep your
eyes and ears open to see where you want a smooth painless transition
and when you want to 'do it with a bang.'

If you look at some of the stuff here
http://blog.explainmydata.com/2012/07/expensive-lessons-in-python-performance.html
you may find that these packages do much of what you want (And add
matplotlib to the set)
This may add some pizzazz to your case.

Warning: In my experience, this can often backfire!