[issue9712] tokenize yield an ERRORTOKEN if the identifier starts with a non-ascii char

[Terry J. Reedy]

Terry J. Reedy <tjreedy at udel.edu> added the comment:

Joshua opened #24194 as a duplicate of this because he could not reopen this.  I am leaving it open as the superseder for this as Serhiy has already added two dependencies there, and because this seems to be a duplicate in turn of #1693050 (which I will close along with #32987).

----------
nosy: +terry.reedy
superseder:  -> tokenize yield an ERRORTOKEN if an identifier uses Other_ID_Start or Other_ID_Continue

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue9712>
_______________________________________