[New-bugs-announce] [issue18596] enable usage of AddressSanitizer in CPython [PATCH]

[halfie]

New submission from halfie:

Hi,

When trying to build CPython "tip" with AddressSanitizer enabled, I get the following crash,

make Parser/pgen
make[1]: Entering directory `/scratch/repos/cpython'
make[1]: `Parser/pgen' is up to date.
make[1]: Leaving directory `/scratch/repos/cpython'
Parser/pgen ./Grammar/Grammar Include/graminit.h Python/graminit.c
=================================================================
==1375== ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60380000f020 at pc 0x40b260 bp 0x7fff6e6cbbf0 sp 0x7fff6e6cbbe8
READ of size 4 at 0x60380000f020 thread T0
    #0 0x40b25f in _PyObject_Realloc /scratch/repos/cpython/Objects/obmalloc.c:1551
    #1 0x403048 in PyNode_AddChild /scratch/repos/cpython/Parser/node.c:98
    #2 0x4037fe in push /scratch/repos/cpython/Parser/parser.c:126
    #3 0x4121bd in parsetok /scratch/repos/cpython/Parser/parsetok.c:211
    #4 0x413344 in PyParser_ParseFileFlags /scratch/repos/cpython/Parser/parsetok.c:86
    #5 0x413422 in getgrammar /scratch/repos/cpython/Parser/pgenmain.c:92
    #6 0x401860 in main /scratch/repos/cpython/Parser/pgenmain.c:55
    #7 0x3cb2221b74 in ?? ??:0
    #8 0x401994 in _start ??:?
0x60380000f020 is located 96 bytes to the left of 0-byte region [0x60380000f080,0x60380000f080)
==1375== AddressSanitizer CHECK failed: ../../../../libsanitizer/asan/asan_allocator2.cc:216 "((id)) != (0)" (0x0, 0x0)
    #0 0x3ab681237d in ?? ??:0
    #1 0x3ab68191b3 in ?? ??:0
    #2 0x3ab680563b in ?? ??:0
    #3 0x3ab6816e8f in ?? ??:0
    #4 0x3ab6817ff1 in ?? ??:0
    #5 0x3ab6812763 in ?? ??:0
    #6 0x40b25f in _PyObject_Realloc /scratch/repos/cpython/Objects/obmalloc.c:1551
    #7 0x403048 in PyNode_AddChild /scratch/repos/cpython/Parser/node.c:98
    #8 0x4037fe in push /scratch/repos/cpython/Parser/parser.c:126
    #9 0x4121bd in parsetok /scratch/repos/cpython/Parser/parsetok.c:211
    #10 0x413344 in PyParser_ParseFileFlags /scratch/repos/cpython/Parser/parsetok.c:86
    #11 0x413422 in getgrammar /scratch/repos/cpython/Parser/pgenmain.c:92
    #12 0x401860 in main /scratch/repos/cpython/Parser/pgenmain.c:55
    #13 0x3cb2221b74 in ?? ??:0
    #14 0x401994 in _start ??:?
make: *** [Include/graminit.h] Error 1

...

AddressSanitizer is a fast memory error detector. See http://clang.llvm.org/docs/AddressSanitizer.html for details.


Build options used
==================

export CFLAGS="-fsanitize=address -g"

export LDFLAGS="-fsanitize=address"

./configure --prefix=/opt/python --with-valgrind


Software Versions
==================

$ hg log

changeset:   84902:6e1dd1ce95b8
branch:      2.7
tag:         tip
parent:      84893:1dbcb0299088
user:        Terry Jan Reedy <tjreedy at udel.edu>
date:        Tue Jul 30 01:37:36 2013 -0400
summary:     Make all idle test case names end with 'Test'.

$ gcc --version
gcc (GCC) 4.8.1 20130603 (Red Hat 4.8.1-1)

$ cat /etc/redhat-release 
Fedora release 19 (Schrödinger’s Cat)

Fix / Work-around
=================

See http://code.google.com/p/address-sanitizer/wiki/AddressSanitizer#Turning_off_instrumentation

Turning off the instrumentation for _PyObject_Free and  _PyObject_Realloc is required.

Advantages
==========

AddressSanitizer can *now* be used to test and debug CPython.

AddressSanitizer has helped me find a large number of bugs in other softwares.

----------
components: Interpreter Core
files: enable-ASAN-in-CPython.diff
keywords: patch
messages: 193917
nosy: halfie
priority: normal
severity: normal
status: open
title: enable usage of AddressSanitizer in CPython [PATCH]
type: crash
versions: Python 3.4
Added file: http://bugs.python.org/file31083/enable-ASAN-in-CPython.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue18596>
_______________________________________