[ python-Bugs-1503789 ] Cannot write source code in UTF16

[SourceForge.net]

Bugs item #1503789, was opened at 2006-06-09 17:38
Message generated for change (Comment added) made by tungwaiyip
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1503789&group_id=5470

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Parser/Compiler
Group: Python 2.4
Status: Open
Resolution: None
Priority: 5
Submitted By: Wai Yip Tung (tungwaiyip)
Assigned to: Nobody/Anonymous (nobody)
Summary: Cannot write source code in UTF16

Initial Comment:
I intend to create some source code in UTF16. I start 
the file with the encoding declaration line:

----------------------------------------------
# -*- coding: UTF-16LE -*-
print "Hello world"
----------------------------------------------

Unfortunately Python does not decode it in UTF16 as 
expected. I have found some language in PEP 0263 that 
says "It does not include encodings which use two or 
more bytes for all characters like e.g. UTF-16." While 
I am disappointed. I accepted this limitation is 
necessary to make keep the parser simple. So my first 
complaint is this fact should be documented in

http://www.python.org/doc/ref/encodings.html

Then I tried to save the source code with BOM. I think 
there should be no excuse not to decode it in UTF16 in 
that case. Unfortunately Python does not support this 
either.

Indeed the only way to get it work is to write the 
encoding declaration line in ASCII and the rest of the 
file in UTF16 (see u16_hybrid.py). Obviously most text 
editor would not support this.

I come up with this because Microsoft adopt UTF16 in 
various places.





----------------------------------------------------------------------

>Comment By: Wai Yip Tung (tungwaiyip)
Date: 2006-06-09 17:39

Message:
Logged In: YES 
user_id=561546




----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=105470&aid=1503789&group_id=5470