[Cython] [cython-users] C++: how to handle failures of 'new'?

[Dag Sverre Seljebotn]

On 07/03/2012 10:39 PM, Dag Sverre Seljebotn wrote:
> On 07/03/2012 09:15 PM, Robert Bradshaw wrote:
>> On Tue, Jul 3, 2012 at 11:43 AM, Dag Sverre Seljebotn
>> <d.s.seljebotn at astro.uio.no> wrote:
>>> On 07/03/2012 08:23 PM, Robert Bradshaw wrote:
>>>>
>>>> On Tue, Jul 3, 2012 at 11:11 AM, Stefan Behnel<stefan_ml at behnel.de>
>>>> wrote:
>>>>>
>>>>> Robert Bradshaw, 03.07.2012 19:58:
>>>>>>
>>>>>> On Tue, Jul 3, 2012 at 9:38 AM, Stefan Behnel wrote:
>>>>>>>
>>>>>>> Dag Sverre Seljebotn, 03.07.2012 18:11:
>>>>>>>>
>>>>>>>> On 07/03/2012 09:14 AM, Stefan Behnel wrote:
>>>>>>>>>
>>>>>>>>> I don't know what happens if a C++ exception is not being
>>>>>>>>> caught, but
>>>>>>>>> I
>>>>>>>>> guess it would simply crash the application. That's a bit more
>>>>>>>>> visible than
>>>>>>>>
>>>>>>>>
>>>>>>>> Yep.
>>>>>>>>
>>>>>>>>> just printing a warning when a Python exception is being
>>>>>>>>> ignored due
>>>>>>>>> to a
>>>>>>>>> missing declaration. It's really unfortunate that our
>>>>>>>>> documentation
>>>>>>>>> didn't
>>>>>>>>> even mention the need for this, because it's not immediately
>>>>>>>>> obvious
>>>>>>>>> that
>>>>>>>>> Cython won't handle errors in "new", and testing for memory errors
>>>>>>>>> isn't
>>>>>>>>> quite what people commonly do in their test suites.
>>>>>>>>>
>>>>>>>>> Apart from that, I agree, users have to take care to properly
>>>>>>>>> declare
>>>>>>>>> the
>>>>>>>>> API they are using.
>>>>>>>>
>>>>>>>>
>>>>>>>> Is there any time you do NOT want a "catch (...) {}" block? I can't
>>>>>>>> see a
>>>>>>>> C++ exception propagating to Python-land doing anything useful
>>>>>>>> ever.
>>>>>>>
>>>>>>>
>>>>>>> That would have been my intuition, too.
>>>>>>
>>>>>>
>>>>>> If it's actually embedded, with the main driver in C++, one might
>>>>>> want
>>>>>> it to propagate up.
>>>>>
>>>>>
>>>>> But what kind of a propagation would that be? On the way out, it could
>>>>> induce anything, from side effects to resource leaks to crashes,
>>>>> depending
>>>>> on what the state of the surrounding code is. It would leave the whole
>>>>> system in an unpredictable state. I cannot imagine anyone really
>>>>> wanting
>>>>> this.
>>>>>
>>>>>
>>>>>>>> So shouldn't we just make --cplus turn *all* external functions and
>>>>>>>> methods
>>>>>>>> (whether C-like or C++-like) into "except +"? (Or keep except+ for
>>>>>>>> manual
>>>>>>>> translation, but always have a catch(...)".
>>>>>>>>
>>>>>>>> Performance overhead is the only reason I can think of to not do
>>>>>>>> this,
>>>>>>>> although IIRC C++ catch blocks are only dealt with during stack
>>>>>>>> unwinds and
>>>>>>>> doesn't cost anything/much (?) when they're not triggered.
>>>>>>>>
>>>>>>>> "except -1" should then actually mean both; "except + except
>>>>>>>> -1". So
>>>>>>>> it's
>>>>>>>> more a question of just adding catch(...) *everywhere*, than making
>>>>>>>> "except
>>>>>>>> +" the default.
>>>>>>>
>>>>>>>
>>>>>>> I have no idea if there is a performance impact, but if there isn't,
>>>>>>> always
>>>>>>> catching all exceptions sounds like a reasonable thing to do. After
>>>>>>> all, we
>>>>>>> have no support for catching C++ exceptions on user side.
>>>>>>
>>>>>>
>>>>>> This is a bit like following every C call with "except *" (though the
>>>>>> performance ratios are unclear). It just seems a lot to wrap every
>>>>>> single line of a non-trivial C++ using function with try..catch
>>>>>> blocks.
>>>
>>>
>>> It seems "a lot" of just what exactly? Generated code? Binary size? Time
>>> spent in GCC parser?
>>
>> All of the above. And we should take a look at the runtime overhead
>> (which is hopefully nil, but who knows.)
>>
>>> Though I guess one might want to try to pull out the try-catch to at
>>> least
>>> only one per code line rather than one per SimpleCallNode.
>>
>> Or even higher, if possible. It's still a lot.
>>
>>> "except *" only has a point when calling functions using the CPython
>>> API,
>>> but most external C functions are pure C, not
>>> CPython-API-using-functions.
>>> OTOH, all external C++ functions are C++ :-)
>>
>> Fair point.
>>
>>> (Also, if we wrote Cython from scratch now I'm pretty sure the
>>> "except *"
>>> defaults would be a tad different.)
>>
>> For sure.
>>
>>>>> But if users are correct about their declarations, we'd end up with
>>>>> the
>>>>> same thing. I think it's worth a try.
>>>>
>>>>
>>>> Most C++ code (that I've ever run into) doesn't use exceptions,
>>>> because exception handling is so broken in C++ anyways.
>>>
>>>
>>> Except for the fact that any code touching "new" could be raising
>>> exceptions? That propagates.
>>
>> I would guess most of the time people don't bother catching these and
>> let the program die, as there's often no sane recovery (the same as
>> MemoryErrors in Python, though I guess C++ is less often used from an
>> event loop).
>
> The point is to provide the user with the stack trace of exactly where
> you ran out of memory. Even if you can't follow it deep into C++-land,
> the Python-side stack trace can be a worth a lot.

Note that I think (I just skimmed gcc documentation) the abort will 
happen the moment you pop back to a C function (unless you compile the C 
code with -fexceptions). I.e. once you're back in the CPython eval loop 
you die, it doesn't propagate back to main(). That means you could crash 
in a lot of callback cases too; your expectation of some kind of 
propagation for callbacks is only true if there's C++ all the way 
(according to my skimming, anyway).

Dag

>
>>
>>> There is a lot of C++ code out there using exceptions. I'd guess that
>>> both
>>> mathematical code and Google-written code is unlike most C++ code out
>>> there
>>> :-) Many C++ programmers go on and on about RAII and auto_ptrs and so
>>> on,
>>> and that doesn't have much point unless you throw an exception now
>>> and then
>>> (OK, there's the occasional return statement where it matters well).
>>
>> True, I've seen a small subset of the C++ code that's out there. Maybe
>> numerical computations use it a lot?
>
> One library I'm using for distributed dense linear algebra (Elemental)
> does. Numerical computations libraries are very different from one another.
>
> Dag
> _______________________________________________
> cython-devel mailing list
> cython-devel at python.org
> http://mail.python.org/mailman/listinfo/cython-devel