Trapping the segfault of a subprocess.Popen

Wed Apr 6 20:27:56 EDT 2011

On Wed, Apr 6, 2011 at 2:20 AM, Pierre GM <pierregmcode at gmail.com> wrote:

> All,
>
> I need to run a third-party binary from a python script and retrieve
> its output (and its error messages). I use something like
> >>> process = subprocess.Popen(options, stdout=subprocess.PIPE,
> stderr=subprocess.PIPE)
> >>> (info_out, info_err) = process.communicate()
> That works fine, except that the third-party binary in question
> doesn't behave very nicely and tend to segfaults without returning any
> error. In that case, `process.communicate` hangs for ever.
>
> I thought about calling a `threading.Timer` that would call
> `process.terminate` if `process.wait` doesn't return after a given
> time... But it's not really a solution: the process in question can
> sometimes take a long time to run, and I wouldn't want to kill a
> process still running.
> I also thought about polling every x s and stopping when the result of
> a subprocess.Popen(["ps","-p",str(initialprocess.pid)],
> stdout=subprocess.PIPE) becomes only the header line, but my script
> needs to run on Windows as well (and no ps over there)...
>
> Any suggestion welcome,
> Thx in advance
> P.
> --
> http://mail.python.org/mailman/listinfo/python-list
>

I'm having a difficult time getting something to hang just from a segfault
alone - my test code is below in shar format.  Are you pretty sure the
process is segfaulting (usually verifiable with
strace/truss/par/trace/etc)?  Based on what you've described, it sounds like
there could be a buffering issue.

Note that Popen objects support poll() and wait() methods.  If you're truly
not sending anything to the subprocess' stdin (I suppose your process could
have had something redirected into it though), I doubt you'd need to worry
about deadlocks much, so you probably could just read in a loop with select
(where the select has a timeout) and a Popen.poll()

#!/bin/sh
# This is a shell archive (produced by GNU sharutils 4.9).
# To extract the files from this archive, save it to some FILE, remove
# everything before the `#!/bin/sh' line above, then type `sh FILE'.
#
lock_dir=_sh09253
# Made on 2011-04-06 17:20 PDT by <dstromberg at benchbox>.
# Source directory was `/home/dstromberg/segfault-subprocess'.
#
# Existing files will *not* be overwritten, unless `-c' is specified.
#
# This shar contains:
# length mode       name
# ------ ---------- ------------------------------------------
#     64 -rw-r--r-- c.c
#     37 -rw-r--r-- Makefile
#    197 -rwxr-xr-x p
#
MD5SUM=${MD5SUM-md5sum}
f=`${MD5SUM} --version | egrep '^md5sum .*(core|text)utils'`
test -n "${f}" && md5check=true || md5check=false
${md5check} || \
  echo 'Note: not verifying md5sums.  Consider installing GNU coreutils.'
save_IFS="${IFS}"
IFS="${IFS}:"
gettext_dir=FAILED
locale_dir=FAILED
first_param="$1"
for dir in $PATH
do
  if test "$gettext_dir" = FAILED && test -f $dir/gettext \
     && ($dir/gettext --version >/dev/null 2>&1)
  then
    case `$dir/gettext --version 2>&1 | sed 1q` in
      *GNU*) gettext_dir=$dir ;;
    esac
  fi
  if test "$locale_dir" = FAILED && test -f $dir/shar \
     && ($dir/shar --print-text-domain-dir >/dev/null 2>&1)
  then
    locale_dir=`$dir/shar --print-text-domain-dir`
  fi
done
IFS="$save_IFS"
if test "$locale_dir" = FAILED || test "$gettext_dir" = FAILED
then
  echo=echo
else
  TEXTDOMAINDIR=$locale_dir
  export TEXTDOMAINDIR
  TEXTDOMAIN=sharutils
  export TEXTDOMAIN
  echo="$gettext_dir/gettext -s"
fi
if (echo "testing\c"; echo 1,2,3) | grep c >/dev/null
then if (echo -n test; echo 1,2,3) | grep n >/dev/null
     then shar_n= shar_c='
'
     else shar_n=-n shar_c= ; fi
else shar_n= shar_c='\c' ; fi
f=shar-touch.$$
st1=200112312359.59
st2=123123592001.59
st2tr=123123592001.5 # old SysV 14-char limit
st3=1231235901

if touch -am -t ${st1} ${f} >/dev/null 2>&1 && \
   test ! -f ${st1} && test -f ${f}; then
  shar_touch='touch -am -t $1$2$3$4$5$6.$7 "$8"'

elif touch -am ${st2} ${f} >/dev/null 2>&1 && \
   test ! -f ${st2} && test ! -f ${st2tr} && test -f ${f}; then
  shar_touch='touch -am $3$4$5$6$1$2.$7 "$8"'

elif touch -am ${st3} ${f} >/dev/null 2>&1 && \
   test ! -f ${st3} && test -f ${f}; then
  shar_touch='touch -am $3$4$5$6$2 "$8"'

else
  shar_touch=:
  echo
  ${echo} 'WARNING: not restoring timestamps.  Consider getting and
installing GNU `touch'\'', distributed in GNU coreutils...'
  echo
fi
rm -f ${st1} ${st2} ${st2tr} ${st3} ${f}
#
if test ! -d ${lock_dir} ; then :
else ${echo} "lock directory ${lock_dir} exists"
     exit 1
fi
if mkdir ${lock_dir}
then ${echo} "x - created lock directory ${lock_dir}."
else ${echo} "x - failed to create lock directory ${lock_dir}."
     exit 1
fi
# ============= c.c ==============
if test -f 'c.c' && test "$first_param" != -c; then
${echo} "x - SKIPPING c.c (file already exists)"
else
${echo} "x - extracting c.c (text)"
  sed 's/^X//' << 'SHAR_EOF' > 'c.c' &&
X
#include <stdio.h>
X
main()
X    {
X    char *p = NULL;
X    *p = '\0';
X    }
X
SHAR_EOF
  (set 20 11 04 06 17 12 05 'c.c'
   eval "${shar_touch}") && \
  chmod 0644 'c.c'
if test $? -ne 0
then ${echo} "restore of c.c failed"
fi
  if ${md5check}
  then (
       ${MD5SUM} -c >/dev/null 2>&1 || ${echo} 'c.c': 'MD5 check failed'
       ) << \SHAR_EOF
bd21e6670ae7a9c711644290c45dc399  c.c
SHAR_EOF
  else
test `LC_ALL=C wc -c < 'c.c'` -ne 64 && \
  ${echo} "restoration warning:  size of 'c.c' is not 64"
  fi
fi
# ============= Makefile ==============
if test -f 'Makefile' && test "$first_param" != -c; then
${echo} "x - SKIPPING Makefile (file already exists)"
else
${echo} "x - extracting Makefile (text)"
  sed 's/^X//' << 'SHAR_EOF' > 'Makefile' &&
X
go: c
X    ./p
X
c: c.c
X    $(CC) -o c c.c
X
SHAR_EOF
  (set 20 11 04 06 17 12 48 'Makefile'
   eval "${shar_touch}") && \
  chmod 0644 'Makefile'
if test $? -ne 0
then ${echo} "restore of Makefile failed"
fi
  if ${md5check}
  then (
       ${MD5SUM} -c >/dev/null 2>&1 || ${echo} 'Makefile': 'MD5 check
failed'
       ) << \SHAR_EOF
9d831f5d17a790719720363996179f44  Makefile
SHAR_EOF
  else
test `LC_ALL=C wc -c < 'Makefile'` -ne 37 && \
  ${echo} "restoration warning:  size of 'Makefile' is not 37"
  fi
fi
# ============= p ==============
if test -f 'p' && test "$first_param" != -c; then
${echo} "x - SKIPPING p (file already exists)"
else
${echo} "x - extracting p (text)"
  sed 's/^X//' << 'SHAR_EOF' > 'p' &&
#!/usr/bin/python
X
import subprocess
X
process = subprocess.Popen('./c', stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(info_out, info_err) = process.communicate('foo')
print(process.returncode)
X
SHAR_EOF
  (set 20 11 04 06 17 19 15 'p'
   eval "${shar_touch}") && \
  chmod 0755 'p'
if test $? -ne 0
then ${echo} "restore of p failed"
fi
  if ${md5check}
  then (
       ${MD5SUM} -c >/dev/null 2>&1 || ${echo} 'p': 'MD5 check failed'
       ) << \SHAR_EOF
4484b44e31ea9c233139f76818aa2183  p
SHAR_EOF
  else
test `LC_ALL=C wc -c < 'p'` -ne 197 && \
  ${echo} "restoration warning:  size of 'p' is not 197"
  fi
fi
if rm -fr ${lock_dir}
then ${echo} "x - removed lock directory ${lock_dir}."
else ${echo} "x - failed to remove lock directory ${lock_dir}."
     exit 1
fi
exit 0
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20110406/16ce5940/attachment-0001.html>