Adding behaviour for managing "task" dependencies

[m.pricejones at gmail.com]

Hi,

I'm currently writing an animation pipeline in Python which is a
system for controlling the flow of work and assets for a team of
people working on a computer animated film. The system will be fairly
large with a database backend.

One particular problem that I'd like to address is the need for
managing dependent tasks. Often you need to process one particular set
of information before starting on another so when you batch them all
together and send them off to a set of computers to process you need
some way of formulating the dependencies. This preferably ends up
looking a lot like a Makefile in the end with syntax a bit like:

task task_name (dependencies):
    commands to complete the task
    ...

Where the dependencies are a list of other tasks that need to be
completed first.

However I'd really like to do it in Python, but I'm thinking I'd might
need to extend Python a bit in order to achieve the new syntax. I've
seen attempts to do this within the Python syntax (Scons and buildIt)
and I'm not a big fan of the way it ends up looking. I've worked at a
place that has written it's own language to handle that sort of thing
but then you end up with a language that is good for that but rubbish
at everything else. Python seems like a good basis if I could tweak it
slightly.

One particular point that interests me is the idea of maintaining
compatibility with Python modules so you still have all the
functionality. This makes me think of the "from __future__ import ..."
statements which, if I understand them correctly, can introduce new
syntax like the with_statement, whilst still maintaining compatibility
with older modules?

Is this correct? Can anyone write a syntax changing module or is
__future__ a hard coded special case? I realise I'll have to get into
the C side of things for this. Are there other approaches to this that
I really should be considering instead?

Any thoughts would be most appreciated, though I would like to stress
that I don't think Python should support the syntax I'm proposing I'd
just like to know if I can extend a copy of it to do that.

Mike