Benefits of unicode identifiers (was: Allow additional separator in identifiers)

[Chris Angelico]

On Fri, Nov 24, 2017 at 8:19 AM, Richard Damon <Richard at damon-family.org> wrote:
> On 11/23/17 2:46 PM, Thomas Jollans wrote:
>>
>> On 23/11/17 19:42, Mikhail V wrote:
>>>
>>> I mean for a real practical situation - for example for an average
>>> Python programmer or someone who seeks a programmer job.
>>> And who does not have a 500-key keyboard,
>>
>> I don't think it's too much to ask for a programmer to have the
>> technology and expertise necessary to type their own language in its
>> proper alphabet.
>
>
> My personal feeling is that the language needs to be fully usable with just
> ASCII, so the - character (HYPHEN/MINUS) is the subtraction/negation
> operator, not an in-name hyphen. This also means the main library should use
> just the ASCII character set.
>
> I do also realize that it could be very useful for programmers who are
> programming with other languages as their native, to be able to use words in
> their native language for their own symbols, and thus useful to use their
> own character sets. Yes, doing so may add difficulty to the programmers, as
> they may need to be switching keyboard layouts (especially if not using a
> LATIN based language), but that is THEIR decision to do so. It also may make
> it harder for outside programmers to hep, but again, that is the teams
> decision to make.
>
> The Unicode Standard provides a fairly good classification of the
> characters, and it would make sense to define that an character that is
> defined as a 'Letter' or a 'Number', and some classes of Punctuation
> (connector and dash) be allowed in identifiers.

That's exactly how Python's identifiers are defined (modulo special
handling of some of the ASCII set, for reasons of backward
compatibility).

> Fully implementing may be more complicated than it is worth. An interim
> simple solution would be just allow ALL (or maybe most, excluding a limited
> number of obvious exceptions) of the characters above the ASCII set, with a
> warning that only those classified as above are promised to remain valid,
> and that other characters, while currently not generating a syntax error,
> may do so in the future. It should also be stated that while currently no
> character normalization is being done, it may be added in the future, so
> identifiers that differ only by code point sequences that are defined as
> being equivalent, might in the future not be distinct.

No, that would be a bad idea; some of those characters are more
logically operators or brackets, and some are most definitely
whitespace. Also, it's easier to *expand* the valid character set than
to *restrict* it, so it's better to start with only those characters
that you know for sure make sense, and then add more later. If the
xid_start and xid_continue classes didn't exist, it might be
reasonable to use "Letter, any" and "Number, any" as substitutes; but
those classes DO exist, so Python uses them.

But broadly speaking, yes; it's not hard to allow a bunch of
characters as part of Python source code. Actual language syntax (eg
keywords) is restricted to ASCII and to those symbols that can easily
be typed on most keyboards, but your identifiers are your business.

ChrisA