Best practice for handling exceptions raised at lower levels?

Skip Montanaro skip.montanaro at gmail.com
Mon Feb 1 19:54:49 EST 2021


I'm curious if there are best practices for handling exceptions raised
by lower level modules and packages. For example, I wrote an
application called polly[1] which constructs a personalized dictionary
from email messages using IMAP4. That dictionary is then used as input
to a password generator based on the famous XKCD 936 comic[2]. The
idea is based on a similar tool Chris Angelico mentioned here several
years ago[3], though as not a D&D player, I didn't have player chat
available as user-specific input from which to construct a dictionary
and chose email instead.

So much for background. After a long period of stasis, I decided to
update polly to Python 3. While the 2-to-3 process was
straightforward, I had problems after the update which looked like
IMAP4 issues. I really didn't want to get knee deep in a pond full of
IMAP alligators, so rather than bookmarking the IMAP4 RFC[4], I cast
about for something higher level than imaplib, eventually settling on
IMAPClient[5]. I thus have a collection of modules and packages:
imapclient, imaplib, ssl, socket, and many other non-network bits.
IMAPClient did what I wanted, for the most part abstracting away IMAP4
details. I still have to craft a couple IMAP4 parameter strings, but
the rest of the interface is pretty Pythonic.

However... Network applications being what they are, hiccups are going
to happen. In the time since I swapped in the imapclient package, I've
also had to catch exceptions raised by lower level modules/packages I
wasn't using directly, discovering them only as they occurred. (Think
of it as lazy exception discovery. I suppose if I was developing the
application for others, I'd have been more proactive about uncovering
the exceptions.) In addition to a couple imapclient exceptions, I also
now need to catch socket.gaierror, ConnectionError and
imaplib.IMAP4.error.

Here's the crux of the problem. Where does responsibility generally
fall for low level exception handling? I don't mean to pick on
IMAPClient. It's just a recent example and got me thinking about the
problem. Is it (generally) the responsibility of the application
author or the package author? This isn't an issue just for network
applications with a number of moving parts (socket, ssl, higher level
packages, etc). It happens in almost all applications or packages,
even if it's just to deal with exceptions raised by internal data
structures.

Skip

[1] https://github.com/smontanaro/polly
[2] https://xkcd.com/936/
[3] https://mail.python.org/pipermail/python-list/2014-August/827854.html
[4] https://tools.ietf.org/html/rfc3501
[5] https://pypi.org/project/IMAPClient/


More information about the Python-list mailing list