How to except the unexpected?

[James Stroud]

Rene Pijlman wrote:
> One of the things I dislike about Java is the need to declare exceptions
> as part of an interface or class definition. But perhaps Java got this
> right...
> 
> I've writen an application that uses urllib2, urlparse, robotparser and
> some other modules in the battery pack. One day my app failed with an
> urllib2.HTTPError. So I catch that. But then I get a urllib2.URLError, so
> I catch that too. The next day, it encounters a urllib2.HTTPError, then a
> IOError, a socket.timeout, httplib.InvalidURL,...
> 
> How do you program robustly with these modules throwing all those
> different (and sometimes undocumented) exceptions at you?
> 
> A catchall seems like a bad idea, since it also catches AttributeErrors
> and other bugs in the program.
> 

The relevant lines of urllib2, for example, look as such:

class URLError(IOError):
class HTTPError(URLError, addinfourl):
class GopherError(URLError):

This suggests that catching URLError should have caught your HTTPError, 
so you might have the chronology backwards above.

E.g.:

py> class BobError(Exception): pass
...
py> class CarolError(BobError): pass
...
py> try:
...   raise CarolError
... except BobError:
...   print 'got it'
...
got it


Now,

% cat httplib.py | grep -e '^\s*class'

produces the following at one point in its output:

class HTTPException(Exception):
class NotConnected(HTTPException):
class InvalidURL(HTTPException):
class UnknownProtocol(HTTPException):
class UnknownTransferEncoding(HTTPException):
class UnimplementedFileMode(HTTPException):
class IncompleteRead(HTTPException):
class ImproperConnectionState(HTTPException):
class CannotSendRequest(ImproperConnectionState):
class CannotSendHeader(ImproperConnectionState):
class ResponseNotReady(ImproperConnectionState):
class BadStatusLine(HTTPException):

Which suggests that "try: except HTTPException:" will be specific enough 
as a catchall for this module.

The following, then, should catch everything you mentioned except the 
socket timeout:

try:
   whatever()
except URLError, HTTPException:
   alternative()

But it seems to me that working with the internet as you are doing is 
fraught with peril anyway.

James