[issue21765] Idle: make 3.x HyperParser work with non-ascii identifiers.

[Terry J. Reedy]

Terry J. Reedy added the comment:

I just noticed that ColorDelegator has
 idprog = re.compile(r"\s+(\w+)", re.S)
which will recognize unicode 'words', if not exactly Python 'identifiers'.

However, UndoDelegator has 
    alphanumeric = string.ascii_letters + string.digits + "_"
which is the same as in Hyperparser. It is used in

    def classify(self, c):
        if c in self.alphanumeric:
            return "alphanumeric"
        if c == "\n":
            return "newline"
        return "punctuation"
and probably does not do what we really want.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21765>
_______________________________________