exceptions (was Re: Perl is worse!)

[Alex Martelli]

"Martijn Faassen" <m.faassen at vet.uu.nl> wrote in message
news:8m3f0o$hn6$1 at newshost.accu.uu.nl...
> Steve Lamb <grey at despair.rpglink.com> wrote:
> > On 30 Jul 2000 21:25:10 GMT, Martijn Faassen <m.faassen at vet.uu.nl>
wrote:
> >>Then have your program catch your exceptions! That's one of the main
> >>reasons for the existence of the exception handling mechanism; to
recover
> >>gracefully from the unexpected.
>
> >>It is possible to catch *all* exceptions, if you're worried about that.
> >>Or whole groups of exceptions, if you like. They're in a hierarchy.
>
> >     However I find that exceptions come up on the damnedest things,
break you
> > out of loops, cause problems all around and don't let you return to
where you
> > were, esp. when you use general catch all exceptions.  As Alex(?) said,
you
> > don't return to where you were so, to me, it seems quite impossible to
be able
> > to catch /all/ exceptions gracefully, ever.

A matter of know-how, both individual (subjective) and collective.  Writing
_really_ good code to deal with totally arbitrary exceptions is quite
possible,
but it's anything but trivial.  And I'm not claiming to have reached such
skill
personally, YET -- I *have* seen it done, and all the pieces make sense, but
learning how to put them all together to perfection is a slower 'maturation'
process.  I suspect the speed of that process is partly determined by the
kind of software you write -- specifically, its quality requirements and the
incentive-structure focusing on them (i.e., are you going to serve time if
you
let a hardware error kill a patient when you could/should have prevented it
with better exception-handling?-).  Purveyors of commercial off-the-shelf
software for professional-quality applications, whose bugs, however, don't
directly lead to the loss of human life are, I surmise, somewhere "in the
middle"
along this scale.

I haven't seen much literature on the subject, except that based on C++,
which may not be ideal reading for programmers using Java, Python, or
whatever else.  Herb Sutter's "Exceptional C++" is a great book for those
who practice C++ in depth, though (it's not ONLY about exceptions, don't
let the title fool you:-); Bobby Schmidt's series of (I've lost count how
many)
articles on the net, starting at
http://msdn.microsoft.com/library/welcome/dsmsdn/deep051099.htm
(use 'show toc' to get the table-of-contents and proceed to further
articles in the series -- I just did and it turns out there's just 17
articles
in the series, it looked like more!) can also be a useful resource.

I would, by the way, really appreciate other bibliography on the subject,
and particularly URLs; surely there must be something out on proper
design for exception-handling in Java, for example, given that language's
popularity and exception-system -- or Eiffel, Oberon, Haskell, whatever.
Not so much about what each language does to support exception
handling, that's only of middling interest, but rather what the designer/
programmer does to USE that support for all it's worth...


> Well, I have lots of working Python software still. :)
>
> In the presence of bugs in your code, you cannot catch all exceptions
> *gracefully*. This is because you do not know where all your bugs are.
> Bugs are not graceful, period.

I don't think exceptions coming from software bugs are as hard to
handle gracefully compared to those coming from _hardware_
malfunction.  If your project is organized to avoid major disasters
if and when the hardware breaks down in various possible ways,
I think you'll get recovery from bugs in certain modules 'just about
for free'.

Basically, I believe, an approach which very often helps is to
structure your work as a series of (possibly-nested, possibly-linked)
_transactions_ at some "appropriate" level of granularity.  Of
course it helps if you can do _real_ transactions, but when you
can't you can often fake it well enough to still help.

And you don't throw away any information that could help debug.
Never.  "Well... hardly ever" -- logging copious state info when the
key problem is that every disk is full to the bursting point (a frequent
underlying cause of exceptions!) is NOT wise:-).


> As pointed out to you before, you seem to overestimate the amount of
> exceptions you need to catch, to handle gracefully or not.
>
> Let's try to classify exceptions:
>
>   1. A test. You expected this exception and check for it, such as in
>
>     try:
>         b = int(b)
>     except ValueError:
>         b = 0

Yep -- something that often stylistically rankles to people with a C++ or
Java background, where exceptions are (supposed to be) meant for
_exceptional_ cases; but quite OK in Python, it seems to me ("it better
to ask forgiveness than permission" may apply: a preliminary attempt
to find out if a certain operation will succeed may well not be as fruitful
and simple as just attempting that operation, and 'ask forgiveness' by
exception-catching if the 'oops, it WASN'T possible' obtains:-).


>   2. Unexpected external conditions you didn't think of (file turned out
to be
>     locked, etc).

No more memory, no more disk, no more <indispensable resource X,
for any X>, server-timeout, network on the brink, floating-point unit has
just turned into a pumpkin, etc, etc.  The meat-and-potatoes of the
exceptions world.  It's not necessarily an issue of not thinking about it,
but of what one can DO about it...


>   3. an assumption about the structure of input that turns out to be
wrong.
>     i.e., a bug.
>
>   4. a mistake in your code. You call a nonexistent function, refer to
>     nonexistent variables, your algorithm is just wrong, etc. i.e., a bug.
>
> Category 1 exceptions are no problem; you handle them. Category 2
exceptions
> are the most tricky. They may hint at a bug (you should have checked
whether
> the file was locked). What *is* sure is that you usually cannot continue
> gracefully; you can't write to the locked file and you need to, or
whatever.
> Ungraceful handling can be implemented with some outer exception handler,
> or it can be accepted that the program can fail under these circumstances.

As long as enough info gets logged to let later analysis determine the root
cause of the anomaly... diagnosing it more immediately (*AND* logging it;
NEVER [well,. hardly ever:-)] let error/anomaly info show up only in some
message to the user, or you'll eventually pay the price when the error gets
incorrectly reported to you and you'll waste time trying to fix was is not
the
broken part...).  I claim that these cases can generally be made graceful
enough, if the quality-requirements demand it.  For example, you cannot
update the locked-file, but you CAN often (and should, when you can) save
elsewhere enough state that whatever mods the user has worked hard to
implement, can later be resurrected and re-applied.  Actually, a log of what
the user has done so far, that's still not applied to the persistent store
(file,
database, whatever) SHOULD be on disk anyway, safely tucked away in
some reasonably safe place, so that if, e.g., electricity suddenly goes
away,
that doesn't mean hours' worth of work are wasted.  So 'graceful recovery'
just means making as sure as feasible that these temporary logs do NOT
get *corrupted*.  ("Snapshot" info about your own computations can also be
saved, if those computations are so expensive that redoing them would be
more hassle than regularly saving them, but that's unusual unless you do
truly heavy processing.)


> If it *does* turn out you can continue gracefully, you can only continue
> gracefully if you catch the exception and handle it, in which case it
> turns into a variety of category 1. You really fixed a bug in that case.
>
> Note that category 1 and category 2 exceptions generally do not happen
> all over your code. You factored out your code well and these things only
> happen in specific places. You don't have to place handlers all over
> the place.

Not if you design properly in terms of transactions...:-).


> Category 3 and 4 are bugs. Bugs cannot be handled gracefully. Again you
> have two options; have the program stop in the face of a bug, or do a
> catch-all and try to continue anyway.

I think that if 2 is truly well-handled, most of 3 and 4 will come "for
free",
or almost so.


> In the light of that, let's translate what seem to be saying:
>
> """
>      However I find that bugs come up on the damnedest things, break you
> out of loops, cause problems all around and don't let you return to where
you
> were [in the algorithm], esp. when you use general catch all exceptions.
> As Alex(?) said, you don't return to where you were so, to me, it seems
quite
> impossible to be able to catch /all/ bugs gracefully, ever.
> """
>
> The answer to that is easy: No, of course not.

Well, ok, not *ALL*.  But you can _design_ for that, to avoid corrupting the
precious persistent data, and avoid wasting a lot of worktime by the user,
under presence of MOST kinds of bugs...


Alex