[I18n-sig] PEP 263 and Japanese native encodings

[Tamito KAJIYAMA]

Hi,

I read the PEP 263: Defining Python Source Code Encodings
(revision 1.9).  Here some comments after a discussion on the
PEP in a Japanese Python mailing list.

First of all, as a Japanese Python programmer, I would like to
use three Japanese native encodings EUC-JP, Shift_JIS and
ISO-2022-JP as a file encoding of Python source files.  I think
these encodings are considered "ASCII compatible" in the sense
you mention in the following paragraph in the "Concepts" section:

  Only ASCII compatible encodings are allowed as source code
  encoding to assure that Python language elements other than
  literals and comments remain readable by ASCII processing tools
  and to avoid problems with wide characters encodings such as
  UTF-16.

However, a participant of the discussion in the Japanese Python
mailing list says, among the three Japanese encodings, Shift_JIS
and ISO-2022-JP are *not* ASCII compatible.  He defines ASCII
compatibility as follows:

  An ASCII compatible encoding (character set) is a superset of
  the ASCII encoding (character set) in which octets from 0x00
  to 0x7f are only used to represent ASCII characters and not
  used in a series of bytes that represent a multibyte character
  (such as Kanji and Hiragana).

This definition is too restrictive IMHO, but anyway the term
"ASCII compatible" is somewhat obscure and needs clarification
since there are at least two interpretations.  For the sake of
the PEP's readers, it's also useful to provide a (partial) list
of encodings that can be used as a file encoding.

In summary, the questions to be raised are:

o What does the term "ASCII compatible" mean?
o Are three Japanese native encodings EUC-JP, Shift_JIS and
  ISO-2022-JP "ASCII compatible"?

Anyway, thank you for the great proposal.  It will enhance the
utility of the language for non-Latin Python programmers once
implemented in the language core.  I really hope that.

Regards,

-- 
KAJIYAMA, Tamito <kajiyama@grad.sccs.chukyo-u.ac.jp>