[issue21765] Idle: make 3.x HyperParser work with non-ascii identifiers.

[Tal Einat]

Tal Einat added the comment:

@Martin:

> 1. This issue is only about identifiers. So processing of
> string literals is technically out of scope.

I added a test with a non-ASCII string literal only for good measure, while I was already adding a test with a non-ASCII identifier. The patch doesn't change anything in that regard. Supporting Unicode string literals could be considered a separate issue, though the root cause of the problems is the same, and fixing it properly will fix both issues.

> 2. I'd suggest to replace .translate with regular expressions:

Indeed, something along those lines should work well, and probably would be fast enough.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue21765>
_______________________________________