Everything you did not want to know about Unicode in Python 3

[alister]

On Tue, 13 May 2014 10:08:42 -0600, Ian Kelly wrote:

> On Tue, May 13, 2014 at 5:19 AM, alister
> <alister.nospam.ware at ntlworld.com> wrote:
>> I am only an amateur python coder which is why I asked if I am missing
>> something
>>
>> I could not see any reason to be using the shutil module if all that
>> the programm is doing is opening a file, reading it & then printing it.
>>
>> is it python that causes the issue, the shutil module or just the OS
>> not liking the data it is being sent?
>>
>> an explanation of why this approach is taken would be much appreciated.
> 
> No, that part is perfectly fine.  This is exactly what the shutil module
> is meant for: providing shell-like operations.  Although in this case
> the copyfileobj function is quite simple (have yourself a look at the
> source -- it just reads from one file and writes to the other in a
> loop), in general the Pythonic thing is to avoid reinventing the wheel.
> 
> And since it's so simple, it shouldn't be hard to see that the use of
> the shutil module has nothing to do with the Unicode woes here.  The
> crux of the issue is that a general-purpose command like cat typically
> can't know the encoding of its input and can't assume anything about it.
> In fact, there may not even be an encoding; cat can be used with binary
> data.  The only non-destructive approach then is to copy the binary data
> straight from the source to the destination with no decoding steps at
> all, and trust the user to ensure that the destination will be able to
> accommodate the source encoding.  Because Python 3 presents stdin and
> stdout as text streams however, it makes them more difficult to use with
> binary data, which is why Armin sets up all that extra code to make sure
> his file objects are binary.

I think I understand that 
in which case I owe Armin an apology, this certainly sounds like a 
failing in pythons handling of stdout



-- 
Get it up, keep it up... LINUX: Viagra for the PC.
   
   -- Chris Abbey