A few questiosn about encoding

[Cameron Simpson]

On 14Jun2013 16:58, Nikos as SuperHost Support <support at superhost.gr> wrote:
| On 14/6/2013 1:14 μμ, Cameron Simpson wrote:
| >Normally a character in a b'...' item represents the byte value
| >matching the character's Unicode ordinal value.
| 
| The only thing that i didn't understood is this line.
| First please tell me what is a byte value

The numeric value stored in a byte. Bytes are just small integers
in the range 0..255; the values available with 8 bits of storage.

| >\x1b is a sequence you find inside strings (and "byte" strings, the
| >b'...' format).
| 
| \x1b is a character(ESC) represented in hex format

Yes.

| b'\x1b' is a byte object that represents what?

An array of 1 byte, whose value is 0x1b or 27.

| >>> chr(27).encode('utf-8')
| b'\x1b'

Transcribing the ESC Unicode character to byte storage.

| >>> b'\x1b'.decode('utf-8')
| '\x1b'

Reading a single byte array containing a 27 and decoding it assuming 'utf-8'.
This obtains a single character string containing the ESC character.

| After decoding it gives the char ESC in hex format
| Shouldn't it result in value 27 which is the ordinal of ESC ?

When printing strings, the non-printable characters in the string are
_represented_ in hex format, so \x1b was printed.

| > No, I mean conceptually, there is no difference between a code-point
| > and its ordinal value. They are the same thing.
| 
| Why Unicode charset doesn't just contain characters, but instead it
| contains a mapping of (characters <--> ordinals) ?

Look, as far as a computer is concerned a character and an ordinal
are the same thing because you just store character ordinals in
memory when you store a string.

When characters are _displayed_, your Terminal (or web browser or
whatever) takes character ordinals and looks them up in a _font_,
which is a mapping of character ordinals to glyphs (character
images), and renders the character image onto your screen.

| I mean what we do is to encode a character like chr(65).encode('utf-8')
| What's the reason of existence of its corresponding ordinal value
| since it doesn't get involved into the encoding process?

Stop thinking of Unicode code points and ordinal values as separate
things. They are effectively two terms for the same thing. So there
is no "corresponding ordinal value". 65 _is_ the ordinal value.

When you run:

  chr(65).encode('utf-8')

you're going:

  chr(65) ==> 'A'
    Producing a string with just one character in it.
    Internally, Python stores an array of character ordinals, thus: [65]

  'A'.encode('utf-8')
    Walk along all the ordinals in the string and transribe them as bytes.
    For 65, the byte encoding in 'utf-8' is a single byte of value 65.
    So you get an array of bytes (a "bytes object" in Python), thus: [65]

-- 
Cameron Simpson <cs at zip.com.au>

The double cam chain setup on the 1980's DOHC CB750 was another one of
Honda's pointless engineering breakthroughs. You know the cycle (if you'll
pardon the pun :-), Wonderful New Feature is introduced with much fanfare,
WNF is fawned over by the press, WNF is copied by the other three Japanese
makers (this step is sometimes optional), and finally, WNF is quietly dropped
by Honda.
        - Blaine Gardner, <blgardne at sim.es.com>