[issue2134] function generate_tokens at tokenize.py yields wrong token for colon

[Guilherme Polo]

Guilherme Polo added the comment:

I'm attaching a patch that solves this and updates tests.

Added file: http://bugs.python.org/file9459/tokenize_r60884.diff

__________________________________
Tracker <report at bugs.python.org>
<http://bugs.python.org/issue2134>
__________________________________