Printing characters outside of the ASCII range

[danielk]

On Friday, November 9, 2012 12:48:05 PM UTC-5, Dave Angel wrote:
> On 11/09/2012 12:17 PM, danielk wrote:
> 
> > I'm converting an application to Python 3. The app works fine on Python 2.
> 
> >
> 
> > Simply put, this simple one-liner:
> 
> >
> 
> > print(chr(254))
> 
> >
> 
> > errors out with:
> 
> >
> 
> > Traceback (most recent call last):
> 
> >   File "D:\home\python\tst.py", line 1, in <module>
> 
> >     print(chr(254))
> 
> >   File "C:\Python33\lib\encodings\cp437.py", line 19, in encode
> 
> >     return codecs.charmap_encode(input,self.errors,encoding_map)[0]
> 
> > UnicodeEncodeError: 'charmap' codec can't encode character '\xfe' in position 0: character maps to <undefined>
> 
> >
> 
> > I'm using this character as a delimiter in my application.
> 
> >
> 
> > What do I have to do to convert this string so that it does not error out?
> 
> 
> 
> What character do you want?  What characters do your console handle
> 
> directly?  What does a "delimiter" mean for your particular console?
> 
> 
> 
> Or are you just printing it for the fun of it, and the real purpose is
> 
> for further processing, which will not go to the console?
> 
> 
> 
> What kind of things will it be separating?  (strings, bytes ?)  Clearly
> 
> you originally picked it as something unlikely to occur in those elements.
> 
> 
> 
> When those things are combined with a separator between, how are the
> 
> results going to be used?  Saved to a file?  Printed to console?  What?
> 
> 
> 
> -- 
> 
> 
> 
> DaveA

The database I'm using stores information as a 3-dimensional array. The delimiters between elements are chr(252), chr(253) and chr(254). So a record can look like this (example only uses one of the delimiters for simplicity):

name + chr(254) + address + chr(254) + city + chr(254) + st + chr(254) + zip

The other delimiters can be embedded within each field. For example, if there were multiple addresses for 'name' then the 'address' field would look like this:

addr1 + chr(253) + addr2 + chr(253) + addr3 + etc ...

I use Python to connect to the database using subprocess.Popen to run a server process. Python requests 'actions' like 'read' and 'write' to the server process, whereby the server process performs the actions. Some actions require that the server send back information in the form of records that contain those delimiters.

I have __str__ and __repr__ methods in the classes but Python is choking on those characters. Surely, I could convert those characters on the server before sending them to Python and that is what I'm probably going to do, so guess I've answered my own question. On Python 2, it just printed the 'extended' ASCII representation.

I guess the question I have is: How do you tell Python to use a specific encoding for 'print' statements when I know there will be characters outside of the ASCII range of 0-127?