Best practice for handling exceptions raised at lower levels?

[Barry Scott]

> On 2 Feb 2021, at 00:54, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> 
> I'm curious if there are best practices for handling exceptions raised
> by lower level modules and packages. For example, I wrote an
> application called polly[1] which constructs a personalized dictionary
> from email messages using IMAP4. That dictionary is then used as input
> to a password generator based on the famous XKCD 936 comic[2]. The
> idea is based on a similar tool Chris Angelico mentioned here several
> years ago[3], though as not a D&D player, I didn't have player chat
> available as user-specific input from which to construct a dictionary
> and chose email instead.
> 
> So much for background. After a long period of stasis, I decided to
> update polly to Python 3. While the 2-to-3 process was
> straightforward, I had problems after the update which looked like
> IMAP4 issues. I really didn't want to get knee deep in a pond full of
> IMAP alligators, so rather than bookmarking the IMAP4 RFC[4], I cast
> about for something higher level than imaplib, eventually settling on
> IMAPClient[5]. I thus have a collection of modules and packages:
> imapclient, imaplib, ssl, socket, and many other non-network bits.
> IMAPClient did what I wanted, for the most part abstracting away IMAP4
> details. I still have to craft a couple IMAP4 parameter strings, but
> the rest of the interface is pretty Pythonic.
> 
> However... Network applications being what they are, hiccups are going
> to happen. In the time since I swapped in the imapclient package, I've
> also had to catch exceptions raised by lower level modules/packages I
> wasn't using directly, discovering them only as they occurred. (Think
> of it as lazy exception discovery. I suppose if I was developing the
> application for others, I'd have been more proactive about uncovering
> the exceptions.) In addition to a couple imapclient exceptions, I also
> now need to catch socket.gaierror, ConnectionError and
> imaplib.IMAP4.error.
> 
> Here's the crux of the problem. Where does responsibility generally
> fall for low level exception handling? I don't mean to pick on
> IMAPClient. It's just a recent example and got me thinking about the
> problem. Is it (generally) the responsibility of the application
> author or the package author? This isn't an issue just for network
> applications with a number of moving parts (socket, ssl, higher level
> packages, etc). It happens in almost all applications or packages,
> even if it's just to deal with exceptions raised by internal data
> structures.

The principle that I code to is "only catch specific exceptions".
And at the top of my app I catch all exceptions and write enough info
to the app log or stderr as appropriate.

When I write packages I aim to trap the exceptions from the lower levels
and convert into a package specific exceptions or document that a low 
level exception can propagate.

With that said how should I deal with exception from packages
that do not define the exceptions they raise?

First I catch the documented exceptions and handle in an app
specific way. For apps that other people will use I'll also do a code
inspection to spot sources of exceptions. In your case its a network
usage that may need a catch of socket.error.

Clearly we cannot impose a requirement to design packages with
a specific set of rules implemented. This is open source after all.

Barry


> 
> Skip
> 
> [1] https://github.com/smontanaro/polly
> [2] https://xkcd.com/936/
> [3] https://mail.python.org/pipermail/python-list/2014-August/827854.html
> [4] https://tools.ietf.org/html/rfc3501
> [5] https://pypi.org/project/IMAPClient/
> -- 
> https://mail.python.org/mailman/listinfo/python-list
>