[issue12486] tokenize module should have a unicode API

[Thomas Kluyver]

Thomas Kluyver <thomas at kluyver.me.uk> added the comment:

Thanks - I had forgotten it, just fixed it now.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue12486>
_______________________________________