[Python-ideas] Interest in seeing sh.py in the stdlib

[Vinay Sajip]

Andrew Moffat <andrew.robert.moffat at ...> writes:

> I'm interested in making sh.py more accessible to help bring Python forward in
> the area of shell scripting, so I'm interested in seeing if sh would be
> suitable for the standard library.  Is there any other interest in something
> like this?

I would agree with others who have replied saying that the approach is cute,
but a little too magical. Disclosure: this is an area of interest for
me, and I maintain a project called sarge [1] which sort of fits in the same
space as pbs/sh. It doesn't have the cute shell-command-as-Python-function
idiom (which, in my view, buys very little readability), but it does aim to
offer some features which (AFAICT) sh doesn't have. I'll just list sarge's
features briefly below, if for no other reason than to show that there are
other contenders worth considering (should there be a consensus that the
stdlib needs batteries in this area).

Sarge improves upon subprocess when:

* You want to use command pipelines, but using subprocess out of the box often
  leads to deadlocks because pipe buffers get filled up.

* You want to use bash-style pipe syntax on Windows, but some Windows shells
  don’t support some of the syntax you want to use, like &&, ||, |& and so on.

* You want to process output from commands in a flexible way, and communicate()
  is not flexible enough for your needs – for example, you need to process
  output a line at a time.

* You want to avoid shell injection problems by having the ability to quote
  your command arguments safely.

* subprocess allows you to let stderr be the same as stdout, but not the other
  way around – and you need to do that.

It offers:

* A simple run command which allows a rich subset of Bash-style shell command
  syntax, but parsed and run by sarge so that you can run identically on
  Windows without cygwin. This includes asynchronous calls (using "&" just as
  in bash).

* The ability to format shell commands with placeholders, such that variables
  are quoted to prevent shell injection attacks.

* The ability to capture output streams without requiring you to program your
  own threads. You just use a Capture object and then you can read from it as
  and when you want. A Capture object can capture the output from multiple
  chained commands.

* Delays in commands (e.g. "sleep") are honoured in asynchronous calls.

I would also concur with others who've pointed out that stdlib maintenance
is a long haul affair. I've been maintaining the logging package for around
10 years now :-)

Regards,

Vinay Sajip

[1] http://sarge.readthedocs.org/