[IPython-dev] iPipe and Hotwire Shell

[Colin Walters]

[originally sent with wrong From:]

Hi,

Recently I came across IPipe
(http://ipython.scipy.org/ipython/ipython/wiki/UsingIPipe), and
noticed a lot of similarity with the HotwirePipe language:
http://code.google.com/p/hotwire-shell/wiki/HotwireArchitecture

Looking at the ipipe.py source, we actually have a number of things we
could share.  First, for Hotwire we had to invent our own extended
types such as File
(http://hotwire-shell.googlecode.com/svn/trunk/hotwire/sysdep/fs.py),
because Python sadly doesn't include one.  For example, on Windows you
want to know whether or not the file is hidden, on modern Linux
systems you want to use GnomeVFS to get extended file metadata like
MIME type and file thumbnail, etc.  Basically we want something a lot
closer to Java's File object.  It looks like ipipe defines a lot of
its own classes like this.

Secondly, iPipe has some operators like "iwalk" which are close to
Hotwire builtins
(http://hotwire-shell.googlecode.com/svn/trunk/hotwire/builtins/walk.py).
 But, sharing these things leads to a the first big distinction
between Hotwire and IPipe:

Hotwire is closer to a "system" shell built on Python, and IPipe due
to its IPython base is more a Python toplevel where you can call out
to the system if you want.  Practically speaking, because I'm more of
an OS developer, I find myself wanting to call out to system binaries
more often than not.  I could use IPython and type "!make", "!sudo rpm
...", etc., but it's much nicer to just type "make" and "sudo rpm
...".  I do prefer Hotwire's approach of just taking over the system
namespace with improved versions, rather than having two worlds
separated by "!".

So, the main difference that is reflected here is that IPython's
operators are really *just* Python generators, while Hotwire builtins
are structured to take Unix-style options and string arguments.  But
the goal is that the Hotwire builtins are just a thin convenience
wrapper around a real API, so we could have Hotwire builtins just call
out to a shared implementation.

There's also some code in Hotwire like the SysBuiltin for executing
subprocesses that would be good to share.  I see IPipe just does
os.popen4(self.cmd).  I remember when I thought it was that
easy...before I discovered the horrors of Unix pseudoterminals just so
you can force subprocesses to be line buffered, and the pitfalls of
processing through /bin/sh.

In the longer term, I was recently thinking about trying to rebase
Hotwire around some of the IPython infrastructure; so the HotwirePipe
input language would be compiled to something like IPipe, which is
itself compiled to Python.  Then if you switch the Hotwire input
language to Python, we'd use IPython to evaluate rather than our own
custom stuff.

Your thoughts appreciated!  Personally I enjoy PyGTK ninjitsu a lot
more than I like maintaining File APIs and Python evaluation hacks, so
I'm hoping we can share some of that =)