Task Engine Framework?

[Adam Tauno Williams]

On Mon, 2010-12-06 at 15:11 -0800, Nate wrote: 
> Hello,
> I'm in the process of developing a task engine / workflow module for
> my Python application and I'm wondering if anyone knows of existing
> code that could be used or adapted.  Since I know that's far too
> generic a question, let me share my goals:
> 1) Support long running operations (think backing up millions of
> files) where:
>    - The operation can be paused (application closed) and the
> operation resumed later.
>    - Individual tasks can be chained, run in parallel, or looped over
> (the workflow part)

We have something like that in OIE (OpenGroupware Integration Engine).
<http://sourceforge.net/projects/coils/>.  These things tend to turn out
to be quite specific [and thus not generic].  But if you have any
questions feel free to ask.  The focus in OIE was the ability to
describe processes in BPML and facilitate process management [creating,
queuing, parking (stopping for later resume) of business / ETL tasks.
Parts of the code aren't especially elegant but it does move a fairly
large amount of data every day.

> 2) Would like to graph each defined operation (task A starts task B
> with parameters... ) for documenting algorithms in Software Design
> Document
> 3) Each individual task in the operation would a self-contained
> class.  I'd imagine implementing its action by defining a doTask()
> method
> Hopefully that's clear.  I just feel like someone must have already
> solved this elegantly.  I greatly enjoy Python and I look forward to
> proving its use as a valuable language for a Masters student even
> though everyone thinks I should use C# :-).