[issue12675] tokenize module happily tokenizes code with syntax errors

[Daniel Urban]

Changes by Daniel Urban <urban.dani+py at gmail.com>:


----------
nosy: +durban

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12675>
_______________________________________