UnicodeDecodeError issue

[Dave Angel]

On 1/9/2013 03:23, Ferrous Cranus wrote:

> Στις 1/9/2013 10:12 πμ, ο/η Chris Angelico έγραψε:
>> On Sun, Sep 1, 2013 at 4:50 PM, Ferrous Cranus <nikos.gr33k at gmail.com> wrote:
>>> Ye i'm aware that i need to define variables before i try to make use of them.
>>> I have study all of your examples and then re-view my code and i can *assure* you that the line statement that tied to set the 'host' variable is very early at the top of the script(of course after imports), and the cur.execute comes after.
>>>
>>> The problem here is not what you say, that i try to drink k a coffee before actually making one first but rather than i cannot drink the coffee although i know *i have tried* to make one first.
>>>
>>>
>>> i will upload the code for you to prove my sayings at pastebin.
>>>
>>> http://pastebin.com/J97guApQ
>>
>> You're setting host inside a try/except block. Are you getting
>> exceptions that prevent it from being set, perhaps?
>>

   <SNIP>
>
> Let alone that i when i try to set the 'host' variable i get this line 
> at my '/tmp/err.out'
>
>
> nikos at superhost.gr [~]# cat /tmp/err.out
> UnicodeDecodeError('utf-8', b'\xb6\xe3\xed\xf9\xf3\xf4\xef 
> \xfc\xed\xef\xec\xe1 \xf3\xf5\xf3\xf4\xde\xec\xe1\xf4\xef\xf2', 0, 1, 
> 'invalid start byte')
>

Let's examine the relevant code from your pastebin above:


try:
        gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
        city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
        host = socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or "Proxy Detected"
except Exception as e:
        print( repr(e), file=open( '/tmp/err.out', 'w' ) )
 
Presumably you do realize that any exception on any of the three lines
will skip the remainder, and print that line to the /tmp/err.out file? 
And that if any exception occurs, it'll be before 'host' is initialized
at all?  Possibly before 'city' is initialized, or even 'gi' ?   So
there's no reason yet to assume that somehow setting the 'host'
variable triggers anything. When in doubt, consider including only one
line in any given try block. But let's try simpler changes.

Your file mode is 'w' which will wipe out anything written earlier.  Yet
if the exception does not happen, the file will still contain error(s)
from some earlier run.  You should open (and truncate) the file before
the first use, then use the same handle in each exception at which you
want to record any information.  Further, you ought to put more into the
file than just the repr(e) value.  For starters, the Traceback would be
nice.

If I were doing it, I'd be printing  repr(sys.exc_info()) to that file. 
I'd also have some kind of label, so that once I had more than one of
these in the code, I'd have an easyt way to tell which was which.

This is my first crack at it (untested):

errout = open("/tmp/err.out", "w")   #opens and truncates the error
output file
try:
    gi = pygeoip.GeoIP('/usr/local/share/GeoIPCity.dat')
    city = gi.time_zone_by_addr( os.environ['REMOTE_ADDR'] ) or
gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
    host =socket.gethostbyaddr( os.environ['REMOTE_ADDR'] )[0] or
socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0] or
"Proxy Detected"
except Exception as e:
    print( "Xyzzy exception-", repr(sys.exc_info()), file=errout)
    errout.flush()


Note that I haven't had to use exc_info() in my own code, so I'm sure it
could be formatted prettier.  But right now, you need to stop throwing
away useful information.


-- 
DaveA