Flexible string representation, unicode, typography, ...

[wxjmfauth at gmail.com]

Le dimanche 26 août 2012 00:26:56 UTC+2, Ian a écrit :
> On Sat, Aug 25, 2012 at 9:47 AM,  <wxjmfauth at gmail.com> wrote:
> 
> > For those you do not know, the go language has introduced
> 
> > the rune type. As far as I know, nobody is complaining, I
> 
> > have not even seen a discussion related to this subject.
> 
> 
> 
> Python has that also.  We call it "int".
> 
> 
> 
> More seriously, strings in Go are not sequences of runes.  They're
> 
> actually arrays of UTF-8 bytes.  That means that they're quite
> 
> efficient for ASCII strings, at the expense of other characters, like
> 
> Chinese (wait, this sounds familiar for some reason).  It also means
> 
> that you have to bend over backwards if you want to work with actual
> 
> runes instead of bytes.  Want to know how many characters are in your
> 
> string?  Don't call len() on it -- that will only tell you how many
> 
> bytes are in it.  Don't try to index or slice it either -- that will
> 
> (accidentally) work for ASCII strings, but for other strings your
> 
> indexes will be wrong.  If you're unlucky you might even split up the
> 
> string in the middle of a character, and now your string has invalid
> 
> characters in it.  The right way to do it looks something like this:
> 
> 
> 
> len([]rune("白鵬翔"))  // get the length of the string in characters
> 
> string([]rune("白鵬翔")[0:2])  // get the substring containing the first
> 
> two characters
> 
> 
> 
> It reminds me of working in Python 2.X, except that instead of an
> 
> actual unicode type you just have arrays of ints.


Sorry, you do not get it.

The rune is an alias for int32. A sequence of runes is a
sequence of int32's. Go do not spend its time in using a
machinery to work with, to differentiate, to keep in memory
this sequence according to the *characers* composing this
"array of code points".

The message is even stronger. Use runes to work comfortably [*]
with unicode:
rune -> int32 -> utf32 -> unicode (the perfect scheme, cann't be
better)

[*] Beyond my skill and my kwowloge and if I understood correctly,
this rune is even technically optimized to ensure it it always
an int32.

len() or slices() have nothing to do here.

My experience with go is equal to uero + epsilon.

jmf