Fwd: Lossless bulletproof conversion to unicode (backslashing)

[Chris Angelico]

On Wed, May 27, 2015 at 9:52 PM, anatoly techtonik <techtonik at gmail.com> wrote:
> And the short answer is that we need unicode because we are printing this
> information to the stdout, and stdout is opened in text mode at least on
> Windows, and without explicit conversion, Python will try to decode stuff
> as being `ascii` and fail anyway.

So you're working with text. That means you HAVE to decode it somehow;
you fundamentally cannot print bytes to the console. Lossless
concealment of arbitrary bytes won't help you. If you can't adequately
decode everything, either backslash-escape the rest, or use a
replacement character; you can't print out those bytes.

And no, I will not cc you. Subscribe to the list if you're going to
ask a question.

ChrisA