[docs] [issue28489] Fix comment in tokenizer.c

[Eric V. Smith]

Eric V. Smith added the comment:

FWIW, in Lib/tokenize.py, it's _all_string_prefixes():

>>> _all_string_prefixes()
set(['', 'FR', 'rB', 'rF', 'BR', 'Fr', 'RF', 'rf', 'RB', 'fr', 'B', 'rb', 'F', 'Br', 'R', 'U', 'br', 'fR', 'b', 'f', 'Rb', 'Rf', 'r', 'u', 'bR'])

My basic point is that trying to list them all is hard and a maintenance problem. So as long as we're not being exhaustive, the comment should just state the gist of what the code does.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue28489>
_______________________________________