[issue16152] Trailing whitespace makes tokenize.generate_tokens pathological

[Łukasz Langa]

Change by Łukasz Langa <lukasz at langa.pl>:


----------
pull_requests: +6273

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue16152>
_______________________________________