New internal string format in 3.3

[wxjmfauth at gmail.com]

Le dimanche 19 août 2012 12:26:44 UTC+2, Chris Angelico a écrit :
> On Sun, Aug 19, 2012 at 8:19 PM,  <wxjmfauth at gmail.com> wrote:
> 
> > This is precicely the weak point of this flexible
> 
> > representation. It uses latin-1 and latin-1 is for
> 
> > most users simply unusable.
> 
> 
> 
> No, it uses Unicode, and as an optimization, attempts to store the
> 
> codepoints in less than four bytes for most strings. The fact that a
> 
> one-byte storage format happens to look like latin-1 is rather
> 
> coincidental.
> 

And this this is the common basic mistake. You do not push your
argumentation far enough. A character may "fall" accidentally in a latin-1.
The problem lies in these european characters, which can not fall in this
coding. This *is* the cause of the negative side effects.
If you are using a correct coding scheme, like cp1252, mac-roman or
iso-8859-15, you will never see such a negative side effect.
Again, the problem is not the result, the encoded character. The critical
part is the character which may cause this side effect.
You should think "character set" and not encoded "code point", considering
this kind of expression has a sense in 8-bits coding scheme.

jmf