[C++-sig] Ownership of a C++ object extended in Python using wrapper

[Jim Bosch]

On Tue, 2010-03-16 at 20:56 +0100, Alexandre Hamez wrote:
> On 16 mars 2010, at 13:12, Neal Becker wrote:
> 
> > Alexandre Hamez wrote:
> > 
> >> Hi all,
> >> 
> >> I have a C++ library that needs to store in a hash map user's code, that
> >> is, Python-extended objects. To make the C++ interface available to
> >> Python, I use boost::python::wrapper, as stated in the tutorial
> >> 
> > (http://www.boost.org/doc/libs/1_42_0/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_virtual_functions).
> >> Because I store this extend objects, I need to tell Python that I've got a
> >> reference on these objects.
> >> A thread from 2007
> >> (http://mail.python.org/pipermail/cplusplus-sig/2007-March/011790.html)
> >> states there is a problem for this particular use case, but in the end, no
> >> solution is given. So my question is the following: as of today, what is
> >> the correct way to  tell Python we hold a reference to Python extend
> >> objects of C++ classes, using the boost::python::wrapper mechanism?
> >> 
> >> Thanks,
> >> ---------------------
> >> Alexandre Hamez
> > 
> 
> Thanks for answering so quickly.
> > I've use 2 methods:
> > 
> > 1) with_custodian_and_ward
> OK, I didn't think of this method. Seems a good way to do it.
> 
> > 2) boost::shared_ptr
> 
> For this point, do you mean you just take the objects that should seen they references increased by a shared_ptr? Just after having asked this question, I tried with this simple solution, and it seems to work. I hope there is no hidden potential crash :-)
> 

If you get a boost::shared_ptr to a Python object (either by wrapping a
function taking a shared_ptr or by using boost::python::extract), that
shared_ptr will properly manage the Python reference count via a custom
deleter that decref's the PyObject* when it goes out of scope.  The only
catch is that shared_ptr doesn't do garbage collection, so you can get
cycles that don't get deleted if your shared_ptrs own other shared_ptrs.
Other than that, you're should be safe if you just use shared_ptr
everywhere.

This will work differently internally if you declare your wrapped
classes to have shared_ptr storage (that's generally what I do), but the
effects should be mostly the same.  Note that you can also declare your
storage to be shared_ptr to a wrapper class.

> Do you use these two methods conjointly in the same project? Or did you identify some use cases which favor a method rather the other one?
> 

I generally use shared_ptr storage for everything, but I often mix that
with with_custodian_and_ward and its cousins (return_internal_reference,
in particular) when I don't have access to a shared_ptr on the C++ side.


Jim Bosch