[capi-sig] Fix random crashes - embedding py/w PyObject pointers to app data.

[Adam Olsen]

On Tue, Oct 21, 2008 at 10:54 PM, Campbell Barton <ideasman42 at gmail.com> wrote:
> On Wed, Oct 22, 2008 at 3:03 PM, Adam Olsen <rhamph at gmail.com> wrote:
>> On Tue, Oct 21, 2008 at 9:34 PM, Campbell Barton <ideasman42 at gmail.com> wrote:
>>> Hi, just spent a couple of hours looking into a bug that others might also have.
>>> If your app embeds python, and allows scripts to throw exceptions
>>> while the application keeps running you may also have this problem.
>>>
>>> sys.last_traceback will store the last exception along with the
>>> namespace dictionary of the script that just ran (with all its
>>> functions etc, variables etc).
>>>
>>> In our case this caused crashes because you could load in new data,
>>> making many of the PyObjects referenced from sys.last_traceback, point
>>> to invalid data. Once sys.last_traceback was set again it would free
>>> the existing data sometimes touching pointers with invalid addresses
>>> causing a segfault.
>>>
>>> Simple solution is to run this after PyErr_Print();
>>>
>>>  PySys_SetObject( "last_traceback", Py_None);
>>>
>>> This can also free memory, if large datasets are references from the
>>> traceback. maybe some of your would fine this useful.
>>
>> Do you mean you have a PyObject with a pointer to malloc'd memory, and
>> after the exception is handled (but not cleared) you free the malloc'd
>> memory, but leave the PyObject with a pointer to it?  If so you should
>> be keeping track of any PyObjects and clearing them instead (or let
>> them take care of freeing the malloc'd memory).
>
> Yeah, this is the case and you're right, we should be aware of the
> PyObjects existence and invalidate them when the memory is freed, or
> let python do the freeing.
> But our application does not center around python, the PyAPI has its
> own directory and PyObjects are not supposed to be used outside that.
> Id be interested in knowing what other apps do - is this even a common problem?

Common in the sense that other apps probably do it right, yes. ;)

How are you calling into python?  If you call a single python function
(which in turn calls many others), and this is what raises the
exception, this is the obvious point to trigger the clearing.



-- 
Adam Olsen, aka Rhamphoryncus