[Cython] [cython-users] C++: how to handle failures of 'new'?

[Stefan Behnel]

[moving this to cython-devel as it's getting technical]

Robert Bradshaw, 28.06.2012 21:46:
> On Thu, Jun 28, 2012 at 11:38 AM, Stefan Behnel wrote:
>> currently, when I write "new CppClass()" in Cython, it generates a straight
>> call to the "new" operator. It doesn't do any error handling. And the
>> current documentation doesn't even mention this case.
>>
>> Is there a "standard" way to handle this? It seems that C++ has different
>> ways to deal with failures here but raises an exception by default. Would
>> you declare the constructor(s) with an "except +MemoryError"? Is there a
>> reason Cython shouldn't be doing this automatically (if nothing else was
>> declared) ?
> 
> I think it certainly makes sense to declare the default constructor as
> "except +" (and std::bad_alloc should become MemoryError),

Right. The code in the constructor can raise other exceptions that must
also be handled properly. An explicit "except +" will handle that.


> but whether
> to implicitly annotate declared constructors is less clear, especially
> as there's no way to un-annotate them.

I agree, but sadly, it's the default behaviour that is wrong. I'm sure we
made lots of users run into this trap already. I fixed the documentation
for now, but the bottom line is that we require users to take care of
proper declarations themselves. Otherwise, the code that we generate is
incorrect, although it's 100% certain that an allocation error can occur,
even if the constructor code doesn't raise any exceptions itself.

Apparently, changing the behaviour of the "new" operator requires a special
annotation "std::nothrow", which then returns NULL on allocation failures.
You can pass that from Cython by hacking up a cname, e.g.

    Rectangle "(std::nothrow) Rectangle" (int w, int h)

I'm sure there are users out there who figured this out (I mean, I did...)
and use it in their code, so I agree that this isn't easy to handle because
Cython simply wouldn't know what the actual error behaviour is for a given
constructor and how to correctly detect an error.

This problem applies only to heap allocation in that form. However, stack
allocation and the new exttype field allocation suffer from similar
problems when the default constructor raises an exception. Exttype fields
are a particularly nasty case because the user has no control over the
allocation. A C++ exception in the C++ class constructor would terminate
the exttype constructor unexpectedly and thus leak resources (in the best
case - no idea how CPython reacts if you throw a C++ exception through its
type instantiation code).

Similarly, a C++ exception in the constructor of a stack allocated object
would then originate from the function entry code and potentially hit the
Python function wrapper etc. Again, potentially leaking resources or worse.

To me, this sounds like we should do something about it. At least for the
implicit calls to the default constructor, we should generate "except +"
code automatically because there is no other way to handle them safely.

Stefan