[issue12149] Segfault in _PyObject_GenericGetAttrWithDict

[Ezio Melotti]

New submission from Ezio Melotti <ezio.melotti at gmail.com>:

The attached patch causes a segfault while running test_urllib:

$ ./python -m test test_urllib
[1/1] test_urllib
<test.test_urllib.FakeHTTPConnection object at 0xb66b0fbc>
<test.test_urllib.FakeHTTPConnection object at 0xb66b76fc>
<test.test_urllib.FakeHTTPConnection object at 0xb66b7a44>
<test.test_urllib.FakeHTTPConnection object at 0xb66b7f84>
<test.test_urllib.FakeHTTPConnection object at 0xb66c2034>
<test.test_urllib.FakeHTTPConnection object at 0xb66c2034>
Fatal Python error: Segmentation fault

Current thread 0xb77de8d0:
  File "/home/wolf/dev/py/py3k/Lib/http/client.py", line 451 in close
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 1119 in do_open
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 1166 in http_open
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 347 in _call_chain
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 387 in _open
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 369 in open
  File "/home/wolf/dev/py/py3k/Lib/urllib/request.py", line 138 in urlopen
  File "/home/wolf/dev/py/py3k/Lib/test/test_urllib.py", line 184 in test_url_fragment
[...]
Segmentation fault

The failure seems to be fairly random, for example if I uncomment the tests at the bottom of test_urllib, the segfault happens only in non-verbose mode.  Also adding a few print()s to debug the issue or making other unrelated changes changes the behavior (e.g. it fails only in verbose mode but not in normal mode).

gdb shows that the failure is in Objects/object.c, at line 981 in the _PyObject_GenericGetAttrWithDict function.  The problem seems to be that 'descr' is invalid and descr->ob_type results in an error:

$ gdb -args ./python -m test test_urllib
GNU gdb (Ubuntu/Linaro 7.2-1ubuntu11) 7.2
[...]
Reading symbols from /home/wolf/dev/py/py3k/python...done.
(gdb) run
Starting program: /home/wolf/dev/py/py3k/python -m test test_urllib
[Thread debugging using libthread_db enabled]
[1/1] test_urllib
[New Thread 0xb77e9b70 (LWP 5227)]
<test.test_urllib.FakeHTTPConnection object at 0xb6ec6dfc>

Program received signal SIGSEGV, Segmentation fault.
0x0805c7d8 in _PyObject_GenericGetAttrWithDict (obj=<FakeSocket(io_refs=1) at remote 0xb6ec31e4>, name='close', dict=0x0) at Objects/object.c:982
982             f = descr->ob_type->tp_descr_get;
(gdb) p *(PyUnicodeObject*)descr
$1 = {ob_base = {_ob_next = 0xdbdbdbdb, _ob_prev = 0xdbdbdbdb, ob_refcnt = -606348324, ob_type = 0xdbdbdbdb}, length = -606348325, str = 0xdbdbdbdb, 
  hash = -606348325, state = -606348325, defenc = <unknown at remote 0xdbdbdbdb>}
(gdb) 

FakeHTTPConnection is a subclass of http.client.HTTPConnection, but I also got segfaults in FakeSocket, which is an io.BytesIO subclass.
FWIW I found this while trying to fix ResourceWarnings.  In #12133 I proposed a patch that fixes a ResourceWarnings (included in segfault.diff), and while trying the same approach in urlretrieve (adding a finally: http_conn.close()) and running test_urllib I got the segfault.
Tested on 3.3 only.

----------
components: Interpreter Core
files: segfault.diff
keywords: patch
messages: 136548
nosy: ezio.melotti, pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: Segfault in _PyObject_GenericGetAttrWithDict
type: crash
versions: Python 3.3
Added file: http://bugs.python.org/file22070/segfault.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12149>
_______________________________________