Python Error Messages

Fri Sep 10 02:16:27 EDT 1999

[jonathon]
> Why are PythonError messages so annoying?

Because the tracebacks you're getting aren't showing the lines of code that
are actually failing.  I can explain why that's happening, but it's obscure:
when Python goes to print a traceback, it uses the platform libc fgets
function to read one line at a time, until the target line number is
reached.  It passes fgets a buffer big enough to hold 1000 characters.
fgets won't actually read a full line if it sees 1000 characters go by
without seeing a newline, so any line of code in your .py file that's longer
than 1000 characters will screw up the line count hopelessly.

There are 7 lines of code in the script more than 1000 characters long, and
that's the source of your traceback woes.  The long lines are at:

Break those lines into non-insane pieces, and the traceback problem will go
away.

> I've been trying to debug a script for the last four days,
> to no avail.

This is a 21,000+ line script, without a class or module of its own, no
testing framework or even hooks, and hard-coded little integers on nearly
every line.  In all honesty, you may well spend the next four months trying
to debug it.

> With two of the four databases I run it on, the script does just what it
> is supposed to.  On the other two databases, it doesn't.

The most likely cause is that the latter databases contain data that the
code doesn't handle correctly, data in a form or combination that simply
never appeared in the former two databases.  That's almost certainly not a
Python problem, but logical errors in the code.

> The current exasperating error message:
>
> Traceback (innermost last)
>   File "mass.py", line 21102 in ?
>          print changed_field, " is record_list[28] "
>   File "mass.py", line 20610 in print_web_page
>          table_prefix = "<tr><td> "
>  TypeError:  illegal argument type for built in operation.

The 3 long lines at 744, 745 & 746 are each getting counted as two lines by
fgets, so the traceback is actually showing you line 20610-3 = 20607.  The
true line 20610 is

   title_name = (title_prefix + country_name + bang_two + scott_data +
                 bang_two + michel_data )

One of title_prefix etc probably isn't a string (mostly likely is an
integer, or None).  IOW, the code is assuming that earlier code bound these
names to strings, but earlier code didn't.

> Now what is infuriating, is that the script has run over this
> particular code sequence at least 4000 times, before giving me
> this error.

That's normal for a data-dependent error.

> Questions:
>
> What would trigger these error messages, when there were no
> apparant problems earlier?

The overwhelmingly most likely cause is errors in the code that didn't show
up before simply because the pattern of data that triggers them didn't show
up before.

> How do I do exception handling that maintains something sane,
> when the script obviously isn't?

I don't mean to be abrasive, but the plain truth is that the script is
already a maintenance nightmare.  Without a rational design you're not going
to patch in robust behavior after the fact.  This script should be 10x
shorter than it is, and 10x faster too -- whoever wrote it was clearly not
an experienced Python programmer, and didn't do any of the things an
experienced one would have done to make recovering from errors pleasant.
Consider cutting your losses -- if you're the author, sorry, but you bit off
more than you're ready to chew.  Strongly consider hiring a Python
consultant to get you on the right track with this (not a plug -- I don't
work for money or love <wink>).

> One thing i have noticed, is that the print statements I put
> in, to try to follow what was happening, tend to indicate
> that the actual problem is more than a couple of lines away
> from the line(s) that are quoted on the screen.

Right; see above.

> ...
> For the curious the other obnoxious error is:
> "IndexError: string index out of range"
>
> Obnoxious, because:
> A) It is supposed to be looking at a list, not a string.

The msg is almost certainly telling you the truth:  you may *expect* it to
be looking at a list, but it's telling you as plain as day that you actually
gave it a string.  That's a great clue, not a plague <wink>.

> B) The actual error is several hundred lines from where
>    the message occurs.

?  Sorry, don't know what that means.

if-you-thought-this-was-fun-try-rewriting-in-perl<wink>-ly y'rs  - tim