[Python-Dev] New Py_UNICODE doc

[M.-A. Lemburg]

Nicholas Bastin wrote:
> On May 7, 2005, at 9:29 AM, Martin v. Löwis wrote:
>>With --enable-unicode=ucs2, Python's Py_UNICODE does *not* start
>>supporting the full Unicode ccs the same way it supports UCS-2.
>>Individual surrogate values remain accessible, and supporting
>>non-BMP characters is left to the application (with the exception
>>of the UTF-8 codec).
> 
> I can't understand what you mean by this.  My point is that if you 
> configure python to support UCS-2, then it SHOULD NOT support surrogate 
> pairs.  Supporting surrogate paris is the purvey of variable width 
> encodings, and UCS-2 is not among them.

Surrogate pairs are only supported by the UTF-8 and UTF-16
codecs (and a few others), not the Python Unicode
implementation itself - this treats surrogate code
points just like any other Unicode code point.

This allows us to be flexible and efficient in the implementation
while guaranteeing the round-trip safety of Unicode data processed
through Python.

Your complaint about the documentation (which started this
thread) is valid.

However, I don't understand all the excitement
about Py_UNICODE: if you don't like the way this Python
typedef works, you are free to interface to Python using
any of the supported encodings using PyUnicode_Encode()
and PyUnicode_Decode(). I'm sure you'll find one that
fits your needs and if not, you can even write your
own codec and register it with Python, e.g. UTF-32
which we currently don't support ;-)

Please upload your doc-patch to SF.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, May 07 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::