[IPython-dev] !commands in the zmq model

[Fernando Perez]

On Sat, Aug 28, 2010 at 12:29 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> ps - for reference, the current implementation is along the lines of:
>
> from __future__ import print_function
>
> import sys
> import pexpect
>
> self = get_ipython()
>
> if 1:
>    def system( cmd):
>        cmd = self.var_expand(cmd, depth=2).strip()
>        sh = '/bin/bash'
>        timeout = 0.05 # seconds
>        pcmd = '%s -c %r' % (sh, cmd)
>        try:
>            child = pexpect.run(pcmd, logfile=sys.stdout)
>        except KeyboardInterrupt:
>            print('\nInterrupted command: %r.' % cmd, file=sys.stderr)

OK, I realized there was a problem with the lovingly simple
pexpect.run() form: *we* get the KeyboardInterrupt, but the subprocess
doesn't.  So while it's better than nothing, it would be nice to let
the subprocess handle SIGINT correctly, since it's quite likely that
it may know what to do.

This more elaborate solution seems like a good compromise to me: we
still kill the subprocess (in case something stubborn just decides to
ignore SIGINT), but first we send it a gentle SIGINT to give it a
chance to act on it:

    def system2( cmd):
        cmd = self.var_expand(cmd, depth=2).strip()
        sh = '/bin/bash'
        timeout = 0.05 # seconds
        pats = [pexpect.TIMEOUT, pexpect.EOF]
        out_size = 0
        try:
            child = pexpect.spawn(sh, ['-c', cmd])
            while True:
                res = child.expect_list(pats, timeout)
                print(child.before[out_size:], end='')
                out_size = len(child.before)
                if res==1:
                    break
        except KeyboardInterrupt:
            out_size = len(child.before)
            child.sendline(chr(3))
            res = child.expect_list(pats, timeout)
            print(child.before[out_size:], end='')
            child.terminate(force=True)

In a bunch of tests using this code:

import time,sys
try:
    for i in range(200):
        print str(i)[-1],
        print >> sys.stderr, '.',
        sys.stdout.flush()
        sys.stderr.flush()
        time.sleep(0.01)
    print 'done!'
except KeyboardInterrupt:
    print 'kbint in sleep loop'

###

it behaves *exactly* like we'd want.  Full async output, correct
interleaving of stdout and stderr, we do see the print from the script
after the SIGINT come out, and then the subprocess terminates.

By putting in there the child.terminate() call after the
.sendline(chr(3)), we ensure that even an ill-behaved thing that does

while True:
  try:
     whatever
  except KeyboardInterrupt:
    pass

can actually be killed from within our shell.

In all my tests this is now working 100%, so I'll clean things up and
will put it in.

And for Windows things will unfortunately be much less satisfactory,
but without some pretty deep expertise on the windows APIs to get
something like pexpect to run there, I think we're stuck.

Cheers,

f