catch UnicodeDecodeError

[jaroslav.dobrek at gmail.com]

On Wednesday, July 25, 2012 1:35:09 PM UTC+2, Philipp Hagemeister wrote:
> Hi Jaroslav,
> 
> you can catch a UnicodeDecodeError just like any other exception. Can
> you provide a full example program that shows your problem?
> 
> This works fine on my system:
> 
> 
> import sys
> open('tmp', 'wb').write(b'\xff\xff')
> try:
>     buf = open('tmp', 'rb').read()
>     buf.decode('utf-8')
> except UnicodeDecodeError as ude:
>     sys.exit("Found a bad char in file " + "tmp")
> 

Thank you. I got it. What I need to do is explicitly decode text.

But I think trial and error with moving files around will in most cases be faster. Usually, such a problem occurs with some (usually complex) program that I wrote quite a long time ago. I don't like editing old and complex programs that work under all normal circumstances.

What I am missing (especially for Python3) is something like:

try:
    for line in sys.stdin:
except UnicodeDecodeError:
    sys.exit("Encoding problem in line " + str(line_number))

I got the point that there is no such thing as encoding-independent lines. But if no line ending can be found, then the file simply has one single line.