creating pipelines in python

[Robert Kern]

per wrote:
> hi all,
> 
> i am looking for a python package to make it easier to create a
> "pipeline" of scripts (all in python). what i do right now is have a
> set of scripts that produce certain files as output, and i simply have
> a "master" script that checks at each stage whether the output of the
> previous script exists, using functions from the os module. this has
> several flaws and i am sure someone has thought of nice abstractions
> for making these kind of wrappers easier to write.
> 
> does anyone have any recommendations for python packages that can do
> this?

You may want to try joblib or ruffus. I haven't had a chance to evaluate either 
one, though.

http://pypi.python.org/pypi/joblib/
http://pypi.python.org/pypi/ruffus/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco