[C++-sig] Proposal to improve Python exception handling in Boost.Python

[David Abrahams]

on Thu Jun 19 2008, Frank Benkstein <benkstein-AT-langer-emv.de> wrote:

> Hi.
>
> Disclaimer:  I'm quite new to Boost.Python, so I might have missed some
> things.  If so, please tell me.  Also I'm currently mostly concerned
> with embedding, not extending.
>
> When embedding Python with Boost.Python it is currently quite
> complicated to handle exceptions originating from Python code.
>
> If a Python error is detected by Boost.Python, error_already_set is
> thrown.  Handling the original Python exception from C++ is cumbersome:
>
> <code>
> using namespace boost::python;
>
> try {
>   object value = some_dict["key"];
>   ...
> } catch (error_already_set &) {
>   if (PyErr_ExceptionMatches(PyExc_KeyError) {
>     // Do actual exception handling
>     ...
>   } else {
>     // We can't just rethrow the exception because that would terminate
>     // the program.
>     PyErr_Print();
>     ...
>   }
> }
> </code>

Yep.

> I would like to see at least the Python standard exceptions [1] to be
> translated to C++ exceptions by Boost.Python (possibly in the
> boost::python::exceptions namespace - mirroring the Python exceptions
> module) so the previous code could be written as follows:

Good idea; I think I always intended to do that.

> <code>
> using namespace boost::python;
>
> try {
>   object value = some_dict["key"];
>   ...
> } catch (exceptions::key_error &e) {
>   BOOST_ASSERT(e.args() == make_tuple("key"));
>   // Do actual exception handling
>   ...
> } catch (exceptions::base_exception &e) {
>   std::cerr << e.traceback << std::flush;
>   std::cerr << e << e.flush;
> }
> </code>
>
> Those exceptions could be automatically translated back to the original
> Python exceptions and the traceback restored (using PyErr_Restore).

Also good... but what is restoring the traceback for?  Doesn't the
traceback that's already there work?

> If you (the Boost.Python developers) agree to this proposal I would
> like to submit patches implementing the mentioned functionality
> through the following steps:
>
> 1. Convert all sites throwing error_already_set or calling
>    throw_error_already_set to calls to handle_python_exception.
>    handle_python_exception would just call throw_error_already_set
>    itself.

We probably ought to package up this little gem, which is a little
bigger than throw_error_already_set, into a single function in the
library:

    if (PyErr_Occurred())
        throw_error_already_set();

> 2. Provide a registry for functions to be called when a specific Python
>    exception is encountered.  
>    These functions get called with three
>    boost::python::object arguments, the exception type, value and the
>    traceback object when an exception is detected by
>    handle_python_exception.  As a fallback if no matching function is
>    found error_already_set would still be thrown.

> 3. Provide a new namespace boost::python::exceptions with C++ exceptions
>    that correspond to the Python standard exceptions[1] including
>    inheritance.  

Good... but boost::python::exceptions::Exception should be derived from
error_already_set for backward compatibility.  Or maybe just

        typedef exceptions::Exception error_already_set;

>    Each class should have a static boost::python::object
>    member "mapped_type" wrapping the Python type object.  

How about "python_type?"

>    And instance members "type", "value" and "traceback" so it can be
>    set or restored as the Python error indicator using PyErr_Restore
>    or PyErr_SetObject if the traceback is None.

Do you really need to store the type in each instance?  Oh, maybe you
do; it could end up being a python exception derived from one of the
standard exceptions.  Then what's "mapped_type" (a.k.a. "python_type")
for?

> 4. Register all exceptions from 3. with the mechanism from 2.  Remove
>    the fallback throwing error_already_set from handle_python_exception
>    because base_exception would match all unknown exceptions.

Why do we want "base_exception," and why have the fallback if you're
just going to remove it?

>    Provide a runtime switch that would make handle_python_exception
>    either use either the registry from 2. or just throw
>    error_already_set.  This is necessary to be compatible to code that
>    is depending on error_already_set.  

I think it's enough to ensure that error_already_set is a base class of
all the things you're throwing.

>    I think it is safe to assume that this switch is guarded by the GIL
>    and therefore thread safe.
>
> As you have probably noticed 3. and 4. will only work with Python >=
> 2.5.  

No, I didn't.  Why do you say that?


-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com