Devanagari int literals [was Re: Should non-security 2.7 bugs be fixed?]

[Terry Reedy]

On 7/19/2015 12:45 AM, Steven D'Aprano wrote:
> On Sun, 19 Jul 2015 01:52 pm, Rustom Mody wrote:

>> JFTR: My kids (um... students) have just managed to add devanagari
>> numerals to python.
>> ie we can now do
>>
>>>>> १ + २
>> 3
>
> That is actually quite awesome, and I would support a new feature that set
> the numeric characters to a particular script, e.g. Latin, Arabic,
> Devanagari, whatever, and printed them in that same script. It seems
> unfortunate that १ + २ prints as 3 rather than ३.
>
> Python already, and has for many years, supported non-ASCII digits in string
> conversions. This is in Python 2.4:
>
> py> int(u'१२')
> 12
> py> float(u'.१२')
> 0.12
>
>
> so the feature goes back a long time.
>
> I think that Python should allow int and float literals using any sequences
> of digits from the same language, e.g. 12 or १२ but not १2.

This could be done easily by adding 10 modified productions from
https://docs.python.org/3/reference/lexical_analysis.html#integer-literals
for each language.  The problem of doing the above in the grammar, 
including the no mixing rule, is that is *would* take a separate set of 
productions for each language supported.

 > One might have
> an interpreter hook which displayed ints and floats using non-ASCII digits,
> or one might even build that function into the intepreter, e.g. have a
> global setting which tells ints and floats what digits to use, e.g.:
>
> sys.setdigits('Devanagari')
>
> I would support this, or something like this, as a language feature. If we
> can write Python using Hindi identifiers, why not Hindi numerals?

As I remember, when non-ascii-digit inputs to int were last discussed 
(python-ideas?, pydev?), the possibility of expanding literals was 
mentioned.  As I remember, to idea was rejected or deferred on the basis 
that nearly all numbers used in production programs are read from files 
as numbers or converted by int or float.  The few numeric literals in 
programs could just as well be converted first, or the int expression 
could be used.

These true observations do not cover the shell, as in the examples 
above.  At some time, Guido has expresses the opinion that interactive 
console python should remain plain and basic and the fancier interaction 
features are the domain of replacement shells.

That brings me, anyway, to Idle.  It currently imitates console python 
in sending keystrokes as is to compile() and output streams as are to 
the tk display.  There are a couple of tracker issues claiming that this 
means that Idle does not imitate console python because a tk display 
does not treat backspace and return the same way simulated terminal 
consoles usually do.

While the bug claims have been rejected, I have been thinking that the 
Idle extension interface could and maybe should be extended so that 
extensions could filter and either transform or act on the input/output 
streams.

With a general mechanism in place, it would be trivial to use 
str.maketrans and str.translate in the input/output streams.  This would 
not disable ascii digit input, including mixtures in a single literal, 
but output has to all be in one digit set.  Selecting a language is a 
somewhat solved problem because last summer we added a extension 
configuration dialog that dynamically generates a dialog tab for each 
extension present.

-- 
Terry Jan Reedy