[issue9969] tokenize: add support for tokenizing 'str' objects

[Nick Coghlan]

Nick Coghlan <ncoghlan at gmail.com> added the comment:

As per Antoine's comment on #9873, requiring a real string via isinstance(source, str) to trigger the string IO version is likely to be cleaner than attempting to duck-type this. Strings are an area where we make so many assumptions about the way their internals work that duck-typing generally isn't all that effective.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue9969>
_______________________________________