pipes.py and its limits

[val]

Oren and All,
    Oren you are great.
    First, coz you are here to help when one needs it.
    Thanx.  (This is the Fantastic List,
    nothing like that in the whole world)
    I spent a couple of weeks torturing the poor
    google trying to locate your 'flows'
    (i had a fuzzy idea that they exist, i tried
    pipeline, meta, sequence, etc but never flows..)
    Anyway, i'm going to follow your pointers and
    study all that material.
thank you very much,
val

----- Original Message -----
From: "Oren Tirosh" <oren-py-l at hishome.net>
Newsgroups: comp.lang.python
Sent: Wednesday, July 03, 2002 10:10 AM
Subject: Re: pipes.py and its limits


> On Wed, Jul 03, 2002 at 09:52:05AM -0400, val wrote:
> > Hi All,
> >    i'm looking at pipes.py, a great pipeline implementation
> >    in python (BTW, who is the author?), and trying to figure
> >    out its limits.  The description says that the commands
> >    to be appended to the pipeline template have to be
> >    shell functions.  I'd like to try to build the dynamic pipelines
> >    using the python functions or callable objects.
> >
> >    In other words, i'd like to modify the pipeline -
> >    its functional components on-the-fly depending on
> >    the results of running the pipeline (its output).
> >
> >    I'm sure the real Pythoneers know how to do that.
> >    Any pointers/help would be highly appreciated.
>
> Take a look at http://tothink.com/python/dataflow for a library that
> implements pipelines (I call them flows) using Python iterators and
> generators.
>
> I also have a library for intergrating external commands more easily
> into Python scripts. http://tothink.com/python/shell
>
> I'm working on merging the two so it will be possible to create flows
> with mixed Python functions and external commands.
>
> I do not intend to support modification of the flow while it is running
> because its elements may be delicate stateful components that have
> different input and output rates, buffering, etc.  Changing it on the
> fly could lose data.  But one of the elements in the flow can change its
> functionality or even embed a separate flow object inside it if you
> want.
>
> Oren
>
>