[New-bugs-announce] [issue25388] tokenizer crash/misbehavior

[Brian Cain]

New submission from Brian Cain:

This issue is similar to (but I believe distinct from) the one reported earlier as http://bugs.python.org/issue24022.  Tokenizer failures strike me as difficult to exploit, but risky nonetheless.

Attached is a test case that illustrates the problem and the output from ASan when it encounters the failure.

All of the versions below that I tested failed in one way or another (segfault, assertion failure, printing enormous blank output to console).  Some fail frequently and some exhibit this failure only occasionally.

Python 3.4.3 (default, Mar 26 2015, 22:03:40) 
Python 2.7.9 (default, Apr  2 2015, 15:33:21) [GCC 4.9.2] on linux2
Python 3.6.0a0 (default:2a8a39640aa2+, Jul  9 2015, 12:28:50) [GCC 4.9.2] on linux

----------
components: Interpreter Core
files: vuln.patch
keywords: patch
messages: 252905
nosy: Brian.Cain
priority: normal
severity: normal
status: open
title: tokenizer crash/misbehavior
versions: Python 2.7, Python 3.4, Python 3.6
Added file: http://bugs.python.org/file40764/vuln.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25388>
_______________________________________