[issue32982] Parse out invisible Unicode characters?

[Franklin? Lee]

New submission from Franklin? Lee <leewangzhong at gmail.com>:

The following line should have a character that trips up the compiler.
  ‎indices = range(5)

The character is \u200e, and was inserted by Google Keep. (I've already reported the issue to Google as a regression.)

Here's the error message:
"""
  File "<stdin>", line 3
    ‎indices = range(5)
           ^
SyntaxError: invalid character in identifier
"""

Depending on the terminal or editor, it may not be possible to tell the problem just from looking. Without knowledge/experience of Unicode, it may not be possible to figure out the problem at all.

Since Python source now uses Unicode by default, should certain invisible characters be stripped out during compilation?

----------
components: Unicode
messages: 313127
nosy: ezio.melotti, leewz, vstinner
priority: normal
severity: normal
status: open
title: Parse out invisible Unicode characters?
type: behavior

_______________________________________
Python tracker <report at bugs.python.org>
<https://bugs.python.org/issue32982>
_______________________________________