[Cython] [cython-users] C++: how to handle failures of 'new'?

[Robert Bradshaw]

On Tue, Jul 3, 2012 at 11:43 AM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 07/03/2012 08:23 PM, Robert Bradshaw wrote:
>>
>> On Tue, Jul 3, 2012 at 11:11 AM, Stefan Behnel<stefan_ml at behnel.de>
>> wrote:
>>>
>>> Robert Bradshaw, 03.07.2012 19:58:
>>>>
>>>> On Tue, Jul 3, 2012 at 9:38 AM, Stefan Behnel wrote:
>>>>>
>>>>> Dag Sverre Seljebotn, 03.07.2012 18:11:
>>>>>>
>>>>>> On 07/03/2012 09:14 AM, Stefan Behnel wrote:
>>>>>>>
>>>>>>> I don't know what happens if a C++ exception is not being caught, but
>>>>>>> I
>>>>>>> guess it would simply crash the application. That's a bit more
>>>>>>> visible than
>>>>>>
>>>>>>
>>>>>> Yep.
>>>>>>
>>>>>>> just printing a warning when a Python exception is being ignored due
>>>>>>> to a
>>>>>>> missing declaration. It's really unfortunate that our documentation
>>>>>>> didn't
>>>>>>> even mention the need for this, because it's not immediately obvious
>>>>>>> that
>>>>>>> Cython won't handle errors in "new", and testing for memory errors
>>>>>>> isn't
>>>>>>> quite what people commonly do in their test suites.
>>>>>>>
>>>>>>> Apart from that, I agree, users have to take care to properly declare
>>>>>>> the
>>>>>>> API they are using.
>>>>>>
>>>>>>
>>>>>> Is there any time you do NOT want a "catch (...) {}" block? I can't
>>>>>> see a
>>>>>> C++ exception propagating to Python-land doing anything useful ever.
>>>>>
>>>>>
>>>>> That would have been my intuition, too.
>>>>
>>>>
>>>> If it's actually embedded, with the main driver in C++, one might want
>>>> it to propagate up.
>>>
>>>
>>> But what kind of a propagation would that be? On the way out, it could
>>> induce anything, from side effects to resource leaks to crashes,
>>> depending
>>> on what the state of the surrounding code is. It would leave the whole
>>> system in an unpredictable state. I cannot imagine anyone really wanting
>>> this.
>>>
>>>
>>>>>> So shouldn't we just make --cplus turn *all* external functions and
>>>>>> methods
>>>>>> (whether C-like or C++-like) into "except +"? (Or keep except+ for
>>>>>> manual
>>>>>> translation, but always have a catch(...)".
>>>>>>
>>>>>> Performance overhead is the only reason I can think of to not do this,
>>>>>> although IIRC C++ catch blocks are only dealt with during stack
>>>>>> unwinds and
>>>>>> doesn't cost anything/much (?) when they're not triggered.
>>>>>>
>>>>>> "except -1" should then actually mean both; "except + except -1". So
>>>>>> it's
>>>>>> more a question of just adding catch(...) *everywhere*, than making
>>>>>> "except
>>>>>> +" the default.
>>>>>
>>>>>
>>>>> I have no idea if there is a performance impact, but if there isn't,
>>>>> always
>>>>> catching all exceptions sounds like a reasonable thing to do. After
>>>>> all, we
>>>>> have no support for catching C++ exceptions on user side.
>>>>
>>>>
>>>> This is a bit like following every C call with "except *" (though the
>>>> performance ratios are unclear). It just seems a lot to wrap every
>>>> single line of a non-trivial C++ using function with try..catch
>>>> blocks.
>
>
> It seems "a lot" of just what exactly? Generated code? Binary size? Time
> spent in GCC parser?

All of the above. And we should take a look at the runtime overhead
(which is hopefully nil, but who knows.)

> Though I guess one might want to try to pull out the try-catch to at least
> only one per code line rather than one per SimpleCallNode.

Or even higher, if possible. It's still a lot.

> "except *" only has a point when calling functions using the CPython API,
> but most external C functions are pure C, not CPython-API-using-functions.
> OTOH, all external C++ functions are C++ :-)

Fair point.

> (Also, if we wrote Cython from scratch now I'm pretty sure the "except *"
> defaults would be a tad different.)

For sure.

>>> But if users are correct about their declarations, we'd end up with the
>>> same thing. I think it's worth a try.
>>
>>
>> Most C++ code (that I've ever run into) doesn't use exceptions,
>> because exception handling is so broken in C++ anyways.
>
>
> Except for the fact that any code touching "new" could be raising
> exceptions? That propagates.

I would guess most of the time people don't bother catching these and
let the program die, as there's often no sane recovery (the same as
MemoryErrors in Python, though I guess C++ is less often used from an
event loop).

> There is a lot of C++ code out there using exceptions. I'd guess that both
> mathematical code and Google-written code is unlike most C++ code out there
> :-) Many C++ programmers go on and on about RAII and auto_ptrs and so on,
> and that doesn't have much point unless you throw an exception now and then
> (OK, there's the occasional return statement where it matters well).

True, I've seen a small subset of the C++ code that's out there. Maybe
numerical computations use it a lot?

+1 to making catch-everywhere a directive at least. Lets see what the
impact is before we decide to make it the default.

- Robert