[Python-Dev] Real segmentation fault handler

Tue Sep 30 04:06:07 CEST 2008

FWIW: I didn't have much luck translating segfaults into exceptions. It (seemed) to work on
some platforms, but not others; this was in the context of C++.
In my experience, it is more useful to generate Python and C stack traces and bail out.
I also do this for floating-point exceptions. The handlers are installed at runtime
from a low-level extension module:

http://cctbx.svn.sourceforge.net/viewvc/cctbx/trunk/boost_adaptbx/meta_ext.cpp?view=markup

Example output is below. It works under Linux and partially under Mac OS X.

Ralf

% boost_adaptbx.segmentation_fault 
Now dereferencing null-pointer ...
show_stack(1): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/segmentation_fault.py(10) run
show_stack(2): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/segmentation_fault.py(14) <module>
libc backtrace (18 frames, most recent call last):
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python [0x4118e9]
  /lib64/libc.so.6(__libc_start_main+0xf4) [0x363241e074]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(Py_Main+0x935) [0x4123c5]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_SimpleFileExFlags+0x1a0) [0x4a8860]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_FileExFlags+0x10e) [0x4a85ce]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCode+0x32) [0x487402]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCodeEx+0x81f) [0x4873bf]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x6bc1) [0x486541]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x2bb9) [0x482539]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyObject_Call+0x13) [0x415ae3]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7c6f7]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x28) [0x2aaaaba87148]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::function0<void>::operator()() const+0x19e) [0x2aaaaba8816e]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7fef8]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0x7d) [0x2aaaaba7fb5d]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<char (*)(char const*), boost::python::default_call_policies, boost::mpl::vector2<char, char const*> > >::operator()(_object*, _object*)+0x29) [0x2aaaab8470a9]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so [0x2aaaab843790]
  /lib64/libc.so.6 [0x3632430f30]
Segmentation fault (Python and libc call stacks above)

% boost_adaptbx.divide_by_zero 
Now dividing by zero (in C++) ...
show_stack(1): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/divide_by_zero.py(10) run
show_stack(2): /net/chevy/raid1/rwgk/dist/boost_adaptbx/command_line/divide_by_zero.py(14) <module>
libc backtrace (18 frames, most recent call last):
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python [0x4118e9]
  /lib64/libc.so.6(__libc_start_main+0xf4) [0x363241e074]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(Py_Main+0x935) [0x4123c5]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_SimpleFileExFlags+0x1a0) [0x4a8860]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyRun_FileExFlags+0x10e) [0x4a85ce]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCode+0x32) [0x487402]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalCodeEx+0x81f) [0x4873bf]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x6bc1) [0x486541]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyEval_EvalFrameEx+0x2bb9) [0x482539]
  /net/chevy/raid1/rwgk/bintbx_py252/base/bin/python(PyObject_Call+0x13) [0x415ae3]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7c6f7]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x28) [0x2aaaaba87148]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::function0<void>::operator()() const+0x19e) [0x2aaaaba8816e]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so [0x2aaaaba7fef8]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0x7d) [0x2aaaaba7fb5d]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so(boost::python::objects::caller_py_function_impl<boost::python::detail::caller<double (*)(double const&, double const&), boost::python::default_call_policies, boost::mpl::vector3<double, double const&, double const&> > >::operator()(_object*, _object*)+0x12a) [0x2aaaab84759a]
  /net/chevy/raid1/rwgk/bintbx_py252/lib/boost_python_meta_ext.so [0x2aaaab8437a4]
  /lib64/libc.so.6 [0x3632430f30]
Floating-point error (Python and libc call stacks above)

----- Original Message ----
From: Victor Stinner <victor.stinner at haypocalc.com>
To: python-dev at python.org
Sent: Monday, September 29, 2008 4:05:53 PM
Subject: [Python-Dev] Real segmentation fault handler

Hi,

I would like to be able to catch SIGSEGV in my Python code! So I started to 
hack Python trunk to support this feature. The idea is to use a signal 
handler which call longjmp(), and add setjmp() at Py_EvalFrameEx() enter.

See attached ("small") patch: segfault.patch

Example read.py with the *evil* ctypes module of invalid memory read:
------------------- 8< --------------
from ctypes import string_at

def fault():
    text = string_at(1, 10)
    print("text = {0!r}".format(text))

def test():
    print("test: 1")
    try:
        fault()
    except MemoryError, err:
        print "ooops!"
        print err

    print("test: 2")
    try:
        fault()
    except MemoryError, err:
        print "ooops!"
        print err

    print("test: end")

def main():
    test()

if __name__ == "__main__":
    main()
------------------- 8< --------------

Result:
------------------- 8< --------------
$ python read.py
test: 1
sizeof()=160
ooops!
segmentation fault
test: 2
sizeof()=160
ooops!
segmentation fault
test: end
------------------- 8< --------------

Example bug1.py of a stack overflow:
----------
loop = None,
for i in xrange(10**5):
    loop = loop, None
----------

Result:
----------
$ python -i bug1.py
>>> print loop
(((((((((...Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
MemoryError: segmentation fault
>>>
----------

Python is able to restore a valid state (stack/heap) after a segmentation 
fault and raise a classical Python exception (I choosed MemoryError, but it 
could be a specific exception).

On my computer (Ubuntu Gutsy/i386), each segfault_frame takes 
sizeof(sigjmpbuf) + sizeof(void*) = 160 bytes, allocated on the stack. I 
don't know if it's huge or not, but that will limit the number of recursive 
calls. The feature can be optional if we add a configure option and some 
#ifdef/#endif. A dedicated stack is needed to be call the signal handler on 
stack overflow error. I choosed 4 KB, but since I only call longjmp(), 
smaller stack might also works.

Does other VM support such feature? JVM, Mono, .NET, etc. ?

I had the idea of catching SIGSEGV after reading the issue 1069092 (stack 
overflow because of too many recursive calls).

-- 
Victor Stinner aka haypo
http://www.haypocalc.com/blog/