[C++-sig] Segfaults in object deallocation

[Alex Leach]

Dear list,

I've started using Boost.Python to wrap a 3rd party C++ library, and  
whilst running my unit tests, the Python interpreter segfault's during  
garbage collection.

With explicit object deletion:

$ gdb -q --args python -c 'import mylib; obj = mylib.MyObj(); del(obj)'
...
*** Error in `/usr/bin/python': free(): invalid next size (fast):  
0x0000000000806fd0 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x7ab06)[0x7ffff74d3b06]
/usr/lib/libc.so.6(+0x7b883)[0x7ffff74d4883]
/usr/lib/libboost_python.so.1.53.0(_ZN5boost6python15instance_holder10deallocateEP7_objectPv+0x15)[0x7ffff004f555]
/usr/lib/libboost_python.so.1.53.0(+0x265a1)[0x7ffff004f5a1]
...
======= Memory map: ========
...


or, leaving it to the garbage collector:-

$ gdb -q --args python -c 'import mylib; obj = mylib.obj() '
...
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b2b0b0 in visit_decref () from /usr/lib/libpython2.7.so.1.0
(gdb)

Quick question: What do I need to do to fix this?

==========================
--- The wrapper class  ---

The wrapper class I've written is fairly simple; I haven't explicitly  
added a 'PyObject* _self' attribute, nor done any of the wiki  
recommendations[3] for managing ownership, as I don't see anywhere in the  
tutorial that recommends any of these methods.

------------------------------------
--- Using boost::python::wrapper ---

The wrapped class (in 3rd party library) does have one virtual method  
(that is protected and I haven't exposed to Python) and a virtual  
destructor. I gather the right thing to do with virtual classes is to also  
inherit from 'boost::python::wrapper<Base>', but this doesn't affect the  
seg-fault. I guess I don't need to inherit from wrapper<Base>, as I am not  
exposing any virtual methods, nor using 'this->get_override(...)' anywhere  
in the class definition.

Another very similar class is in the same source file though (virtual  
destructor and one unexposed virtual method), but this one has a public  
copy constructor. If I inherit from 'wrapper<Base>', I get a compilation  
error regarding an argument mismatch when Boost.Python passes the derived  
class to the copy constructor of the base class. Of course I can add  
noncopyable, to the class_<..> registration, but I'd like the exposed  
class to stay copyable, and I might like to use some of the wrapper  
template class' functionality. Should I wrap the copy constructor in the  
wrapper class too? Would it be possible to add this functionality to the  
wrapper<> template, when the class is not noncopyable?


------------------------------------
--- Calling thread-safe methods  ---

I don't think this is directly relevant, but the classes in question use  
the 3rd party library's own mutex implementations to ensure thread-safety.  
I've just got these working by writing a simple utility class that  
releases the GIL in the constructor and reacquires it in its destructor,  
using the C-Python API PyGILState_* functions[1].
The general tactic was described on the boost.python wiki[2], but the  
PyEval_(Save|Restore)Thread functions described caused the same assertion  
errors that I was getting earlier, with the pthreads mutex. This happened  
when the 3rd party library tried to lock the global mutex.

Although this is mentioned in the C-Python manual, I would have found it  
useful if thread-safety was specifically mentioned in the tutorial or  
reference manual somewhere... I attach the solution I wrote to this  
problem, in case someone would want to add it and document it in  
Boost.Python somewhere.

Usage:

  {
      MakeThreadsafe scope; //< GIL acquired
      // call function that acquires pthreads mutex.
  } //< GIL released when leave scope


------------------------------------
--- Garbage collection segfaul   ---

Any help with this would be really appreciated! I thought that registering  
a wrapped class with class_<...> was enough, to ensure an object's ref  
count was incremented and decremented properly. I guess that's the  
fundamentals of the problem, but am not sure the easiest / best / most  
efficient way of solving the problem.. Again, some specific documentation,  
on when and why this problem occurs and how to solve it, in the tutorial  
would be great!

Looking forward to hearing back...

Kind regards,
Alex


[1] - http://docs.python.org/2.7/c-api/init.html#non-python-created-threads
[2] -  
http://wiki.python.org/moin/boost.python/HowTo#Multithreading_Support_for_my_function
[3] -  
http://wiki.python.org/moin/boost.python/HowTo#ownership_of_C.2B-.2B-_object

-- 
Using Opera's mail client: http://www.opera.com/mail/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: make_threadsafe.hpp
Type: application/octet-stream
Size: 977 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/dcfdacf3/attachment.obj>