GeneratorExit should derive from BaseException, not Exception

[Chad Austin]

Hi Terry,

Thank you for your feedback.  Responses inline:

Terry Reedy wrote:
> "Chad Austin" <chad at imvu.com> wrote in message 
> news:46CA0EFE.6020103 at imvu.com...
> || try:
> | result = yield chatGateway.checkForInvite({'userId': userId})
> | logger.info('checkForInvite2 returned %s', result)
> 
> would not
> except GeneratorExit: <do whatever>
> solve your problem?

Yes, we could add an "except GeneratorExit: raise" clause to every place
we currently catch Exception, but I feel like this is one of those
things where it's hard to get it right in all places and also hard to
cover with unit tests.  Instead, we'll have subtle bugs where finally
clauses don't run because the GeneratorExit was swallowed.

Also, SystemExit and KeyboardInterrupt were made into BaseExceptions for
the same reasons as I'm giving.  (As I understand it, anyway.)

> | except Exception:
> 
> Such catchalls are known to be prone to catch too much
> and are therefore not encouraged ;-).
> As in 'use at your own risk'.
> Guido encourages specific catches just for the reasons you give here.

More below:

> There was a *long* discussion of the current 2.5 exception hierarchy on 
> pydev.  Search either python.org's or gmane's archive if you want to pursue 
> this.  But I expect the people involved would say much the same as above.

I've actually read the background on the exception hierarchy (and agree
with it all), especially other suggestions that GeneratorExit derive
from BaseException.  As I understand it, Guido's objections are threefold:

1) The previous "generators as coroutines" examples were too
theoretical:  I've wanted GeneratorExit to derive from BaseException for
months now, but didn't write this proposal until I actually wrote code
that failed in the presence of task cancellation.

2) You should avoid catching everything with except Exception:  I think
that's too idealistic. Just do a search for try: except: through
publicly available Python.  :)  Sometimes, you really _do_ want to catch
everything.  When you're making a network request that involves
xmlrpclib, urllib2, httplib, etc. you don't actually care what the error
was.  (Well, except that the exceptions are submitted for automated
analysis.)  Similarly, when loading a cache file with pickle, I don't
care what went wrong, because it's not critical and should not be turned
into a crash for the user.  (We automatically report exceptions that
bubble into the main loop as crashes.)

3) If GeneratorExit escapes from the generator somehow and gets raised
in the main loop, then it will bubble out of the application like
SystemExit and KeyboardInterrupt would:  I think this argument is
somewhat specious, because I can't imagine how that would happen.  You'd
have to store exceptions in your generator and explicitly bubble them
out somehow.  Our crash handling has to specially handle
KeyboardInterrupt and SystemExit anyway, since there are currently
non-Exception exceptions, such as strings and custom classes that forgot
to derive from Exception, that should count as crashes.

I personally can't think of any cases where I would _want_ to handle
GeneratorExit.  I just want finally: and with: clauses to do the right
thing when a task is cancelled.  Anyway, I haven't yet encountered any
serious bugs due to this yet...  I'm just worried that if a task is
holding some resource and blocking on something, then the resource won't
get released.  If this really does come up, then I do have a little bit
of python + ctypes that replaces GeneratorExit with ImvuGeneratorExit
(deriving from BaseException), but that's not very appealing.

Thanks again,

-- 
Chad Austin
http://imvu.com/technology