Logging library unicode problem

[Patrol Sun]

What's your system? Simple Chinese Windows???

2008/8/13 Victor Lin <Bornstub at gmail.com>

> Hi,
> I'm writting a application using python standard logging system. I
> encounter some problem with unicode message passed to logging library.
> I found that unicode message will be messed up by logging handler.
>
> piese of StreamHandler:
>
>                try:
>                    self.stream.write(fs % msg)
>                except UnicodeError:
>                    self.stream.write(fs % msg.encode("UTF-8"))
>
> It just write the message to stream. If there is some unicode error,
> it would rewrite msg with utf8 encoding.
>
> I write some code to try:
>
>    import sys
>    print u'ÖÐÎÄ×ÖœyÔ‡'
>    print sys.stdout.encoding
>    sys.stdout.write(u'ÖÐÎÄ')
>
> result of that program:
>
> ÖÐÎÄ×ÖœyÔ‡
> cp950
> Traceback (most recent call last):
>  File "update_stockprice.py", line 92, in <module>
>    sys.stdout.write(u'ã–Øà?')
> UnicodeEncodeError: 'ascii' codec can't encode characters in position
> 0-1: ordin
> al not in range(128)
>
> It show that....
>
> 1. print statement encode what it get with stream.encoding?
> 2. stream.write don't do anything like encoding, just write it
> (because it might be binary data?)
>
> So the problem is : the StreamHandler of standard logging library use
> stream.write to log message, if there is unicode error, unicode string
> will be encode to utf8. This behavior mess my unicode up.
>
> Here I modify the code of StreamHandler:
>
>                try:
>                    print >> self.stream, msg
>                    #self.stream.write(fs % msg)
>                except UnicodeError:
>                    self.stream.write(fs % msg.encode("UTF-8"))
>
> I replace stream.write with print statement, so that it will try to
> use stream.encoding to encode msg. Now everything works fine.
>
> My question is :
> Could the behavior of StreamHandler be considered as a bug?
> If it is, how to report this bug?
> Is my solution correct?
> Are there any side effect will caused by doing so?
> If the code I write is fine, and solve that problem, how to report it
> to Python's project?
> I think this could be helpful for people who also encountered this
> problem.
>
> Thanks.
> Victor Lin.
> --
> http://mail.python.org/mailman/listinfo/python-list
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20080813/71f173a5/attachment-0001.html>