[Patches] [ python-Patches-534304 ] PEP 263 phase 2 Implementation
noreply@sourceforge.net
noreply@sourceforge.net
Sun, 31 Mar 2002 08:16:08 -0800
Patches item #534304, was opened at 2002-03-24 22:52
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=534304&group_id=5470
Category: Parser/Compiler
Group: Python 2.3
Status: Open
Resolution: None
Priority: 5
Submitted By: SUZUKI Hisao (suzuki_hisao)
Assigned to: Nobody/Anonymous (nobody)
Summary: PEP 263 phase 2 Implementation
Initial Comment:
This is a sample implementation of PEP 263 phase 2.
This implementation behaves just as normal Python does
if no other coding hints are given. Thus it does not
hurt anyone who uses Python now. Note that it is
strictly compatible with the PEP in that every program
valid in the PEP is also valid in this implementation.
This implementation also accepts files in UTF-16 with
BOM. They are read as UTF-8 internally. Please try
"utf16sample.py" included.
----------------------------------------------------------------------
>Comment By: SUZUKI Hisao (suzuki_hisao)
Date: 2002-04-01 01:16
Message:
Logged In: YES
user_id=495142
Thank you for your review.
Now 1. and 3. are fixed, and 2. is improved.
(4. is not true.)
----------------------------------------------------------------------
Comment By: Michael Hudson (mwh)
Date: 2002-03-30 20:27
Message:
Logged In: YES
user_id=6656
Not going into 2.2.x.
----------------------------------------------------------------------
Comment By: Martin v. Löwis (loewis)
Date: 2002-03-25 22:23
Message:
Logged In: YES
user_id=21627
The patch looks good, but needs a number of improvements.
1. I have problems building this code. When trying to build
pgen, I get an error message of
Parser/parsetok.c: In function `parsetok':
Parser/parsetok.c:175: `encoding_decl' undeclared
The problem here is that graminit.h hasn't been built yet,
but parsetok refers to the symbol.
2. For some reason, error printing for incorrect encodings
does not work - it appears that it prints the wrong line in
the traceback.
3. The escape processing in Unicode literals is incorrect.
For example, u"\<non-ascii character>" should denote only
the non-ascii character. However, your implementation
replaces the non-ASCII character with \u<hex>, resulting in
\u<hex>, so the first backslash unescapes the second one.
4. I believe the escape processing in byte strings is also
incorrect for encodings that allow \ in the second byte.
Before processing escape characters, you convert back into
the source encoding. If this produces a backslash character,
escape processing will misinterpret that byte as an escape
character.
----------------------------------------------------------------------
You can respond by visiting:
http://sourceforge.net/tracker/?func=detail&atid=305470&aid=534304&group_id=5470