General Purpose Pipeline library?

[Marko Rauhamaa]

ram at zedat.fu-berlin.de (Stefan Ram):

> Jason <jasonhihn at gmail.com> writes:
>>I feel like I'm reinventing a wheel here.  I was wondering if
>>there's already something that exists?
>
>   Why do you want this?

Some time back Stephen D'Aprano demonstrated how the | operator can be
defined to create pipelines in Python. As a hobby project, I started
developing the idea further into a Python-based shell (I call it
"snake"). I kinda proved to myself that it is very much doable and left
it at that.

For example:

   $ ./snake 
   >>> ls()
   notes.txt
   .git
   snake
   notes.txt~
   >>> ls() | cat()
   notes.txt
   .git
   snake
   notes.txt~
   >>> ls() | grep(lambda x: "n" in x)
   notes.txt
   snake
   notes.txt~
   >>> sleep(5)
   >>> sleep(5).bg()
   29766
   >>> 
   [29766] Done: sleep(5)
   >>> X("/bin/echo hello")
   hello
   >>> X("/bin/seq 20") | grep(lambda x: "2" in x)
   2
   12
   20
   >>> 

So snake is just a regular Python REPL with some predefined things that
implement a full-fledged Unix shell, courtesy of the amazingly complete
Linux system call support by Python.

The pipelines relay byte streams, line sequences or JSON arrays. The
pipeline processors are either generators or processes.

>  Can't you just map what you want to do to plain-old Python?

The above is plain Python, but it might be more pythonesque to do the
pipelining using the dot notation:

   feed(dataset).f().g().h().output()


Marko