Unicode blues in Python3

nn pruebauno at latinmail.com
Tue Mar 23 14:46:33 EDT 2010



Gary Herron wrote:
> nn wrote:
> > I know that unicode is the way to go in Python 3.1, but it is getting
> > in my way right now in my Unix scripts. How do I write a chr(253) to a
> > file?
> >
>
> Python3 make a distinction between bytes and string(i.e., unicode)
> types, and you are still thinking in the Python2 mode that does *NOT*
> make such a distinction.  What you appear to want is to write a
> particular byte to a file -- so use the bytes type and a file open in
> binary mode:
>
>  >>> b=bytes([253])
>  >>> f = open("abc", 'wb')
>  >>> f.write(b)
> 1
>  >>> f.close()
>
> On unix (at least), the "od" program can verify the contents is correct:
>  > od abc -d
> 0000000   253
> 0000001
>
>
> Hope that helps.
>
> Gary Herron
>
>
>
> > #nntst2.py
> > import sys,codecs
> > mychar=chr(253)
> > print(sys.stdout.encoding)
> > print(mychar)
> >
> >  > ./nntst2.py
> > ISO8859-1
> > ý
> >
> >  > ./nntst2.py >nnout2
> > Traceback (most recent call last):
> >   File "./nntst2.py", line 5, in <module>
> >     print(mychar)
> > UnicodeEncodeError: 'ascii' codec can't encode character '\xfd' in
> > position 0: ordinal not in range(128)
> >
> >
> >> cat nnout2
> >>
> > ascii
> >
> > ..Oh great!
> >
> > ok lets try this:
> > #nntst3.py
> > import sys,codecs
> > mychar=chr(253)
> > print(sys.stdout.encoding)
> > print(mychar.encode('latin1'))
> >
> >
> >> ./nntst3.py
> >>
> > ISO8859-1
> > b'\xfd'
> >
> >
> >> ./nntst3.py >nnout3
> >>
> >
> >
> >> cat nnout3
> >>
> > ascii
> > b'\xfd'
> >
> > ..Eh... not what I want really.
> >
> > #nntst4.py
> > import sys,codecs
> > mychar=chr(253)
> > print(sys.stdout.encoding)
> > sys.stdout=codecs.getwriter("latin1")(sys.stdout)
> > print(mychar)
> >
> >  > ./nntst4.py
> > ISO8859-1
> > Traceback (most recent call last):
> >   File "./nntst4.py", line 6, in <module>
> >     print(mychar)
> >   File "Python-3.1.2/Lib/codecs.py", line 356, in write
> >     self.stream.write(data)
> > TypeError: must be str, not bytes
> >
> > ..OK, this is not working either.
> >
> > Is there any way to write a value 253 to standard output?
> >

Actually what I want is to write a particular byte to standard output,
and I want this to work regardless of where that output gets sent to.
I am aware that I could do
open('nnout','w',encoding='latin1').write(mychar) but I am porting a
python2 program and don't want to rewrite everything that uses that
script.



More information about the Python-list mailing list