Why BOM in logging message?

[Terry Reedy]

On 1/10/2013 11:06 AM, Roy Smith wrote:

> Well, this is fascinating.  It turns out that while all of our
> machines report that they're running 2.7.3, they have two different
> versions of /usr/lib/python2.7/logging/handlers.py!
>
> -rw-r--r-- 1 root root 45076 Aug  1 05:39 /usr/lib/python2.7/logging/handlers.py
> -rw-r--r-- 1 root root 45143 Apr 20  2012 /usr/lib/python2.7/logging/handlers.py
>
> The April 24th version has the BOM code in SysLogHander.emit():

The 'BOM' you are discussing here is not a true 2 or 4 byte 
byte-order-mark, but the pseudo-BOM (pseudo because a stream of single 
bytes has no byte order within the single bytes) that Micro$tupid adds 
(sometimes) to utf-8 encoded bytes to mark their encoding as utf-8 
rather than anything else. In otherwords, it is a non-(anti-)standard 
U(nicode)E(ncoding)M(ark). It is actually the utf-8 encoding of the 
2-byte BOM, and hence not a single mark! It is really confusing when 
people use 'BOM' to refer to a UEM sequence instead of a BOM.

> |         # Message is a string. Convert to bytes as required by RFC 5424
> |        if type(msg) is unicode:
> |            msg = msg.encode('utf-8')
> |            if codecs:
> |                msg = codecs.BOM_UTF8 + msg
> |        msg = prio + msg

2.7.3 was released in April.

> and the August 1st version doesn't:
>
> |        # Message is a string. Convert to bytes as required by RFC 5424
> |        if type(msg) is unicode:
> |            msg = msg.encode('utf-8')
> |        msg = prio + msg

The issue referenced in an earlier messaged was to remove the UEM where 
it did not belong.

> Is it possible there's two different 2.7.3 builds that somehow got
> packaged by Ubuntu at different times?

As you discovered, Ubuntu sometimes updates a release with bug patches 
before we release a new official bugfix release. 2.7.4, with many 
bugfixes, is still to see the light of day.

> The pattern of which machines
> have the August code and which have the April code correlates with
> when we rolled out each server instance.

Great detective work ;-).

-- 
Terry Jan Reedy