[issue25311] Add f-string support to tokenize.py

[Eric V. Smith]

Eric V. Smith added the comment:

This patch cleans up string matching in tokenize.py, and adds f-string support.

----------
Added file: http://bugs.python.org/file40821/issue25311-1.diff

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue25311>
_______________________________________