[Python-ideas] Processing surrogates in

[random832 at fastmail.us]

On Thu, May 7, 2015, at 18:30, Stephen J. Turnbull wrote:
> Chris Barker writes:
> 
>  > I've read many of the rants about UTF-16, but in fact, it's really
>  > not any worse than UTF-8
> 
> Yes, it is.  It's not ASCII compatible.  You can safely use the usual
> libc string APIs on UTF-8 (except for any that might return only part
> of a string), but not on UTF-16 (nulls).  This is a pretty big
> advantage for UTF-8 in practice.

If you're using libc, why shouldn't you be using the native wide
character types (whether that it UTF-16 or UCS-4) and using the wide
string APIs?