[C++-sig] Boost::Python Reference Counting

Tue May 20 21:26:37 CEST 2014

In your code, after the call del(container), Element::_container is a
dangling pointer. Using

.def("getElement", &Container::getElement, return_internal_reference<>())

should prevent the deletion of container for as long as element
variable will be alive, me understands (ties lifetime of _conteiner
with Element's this). This is of course inconsistent with behaviour of
your C++ code. One may ask a question why would anyone make her code
inconsistent. The tutorial gives one argument:
"Python users should not be able to crash the system just by using our
C++ interface".
The other is, if that code had been written in Python entirely, it
would run perfectly fine. And call policies help us keep our C++ code
intact in such cases (no need for smart pointers).

I would like to hear some expert on that, though.

2014-05-20 17:36 GMT+02:00 Jaedyn Draper <jaedyn.cppsig at jaedyn.co>:
> Upon a bit of further reflection on this topic, I realized my suggestion is
> doing something silly.
> struct Element
> {
>     Element(const Container * c, boost::python::object obj) : _container(c),
> containerPyObj(obj) { }
>
>     std::string name() const;
>     const Container * _container;
>     boost::python::object containerPyObj;
>
> };
>
> Element GetElementProxy(boost::python::object self)
> {
>     Container * container = boost::python::extract<Container *>(self);
>     return Element(container, self);
> }
>
> boost::python::object will handle the incref and decref for you if it's a
> member of that class. No need to do it manually.
>
>
> On 5/20/2014 9:48 AM, Jaedyn Draper wrote:
>>
>> std::shared_ptr and similar types can also be used like so:
>>
>> boost::python::class_<MyClass, std::shared_ptr<MyClass>>
>>
>> But I don't think that would solve the problem because it's a raw pointer
>> to _container being held here. I don't think there's a way to do what you
>> want with raw pointers. The only way I would know of to do that is to make
>>
>>     const Container * _container;
>>
>> into
>>
>>     const std::shared_ptr<Container> _container;
>>
>> or
>>
>>     const boost::shared_ptr<Container> _container;
>>
>> or you can probably make this work with your own custom type, too, by
>> replacing the second argument to class_ with your C++ ref-counted shared
>> pointer class. Then anything exposed to python that deals with the Container
>> class (like the constructor for Element) should return it or accept it as a
>> shared_ptr instead of a pointer.
>>
>> With raw pointers the ownership of the memory sits in just one place, and
>> you're deleting that place. This is an issue that goes deeper than python,
>> down to the C++ architecture - if you did the same thing in C++ using raw
>> pointers like that, you'd have the same behavior. In order to get this sort
>> of ref counting behavior in python, you have to have it in C++.
>>
>> In general, unless you're working with very performance-sensitive code
>> like a rendering pipeline (which isn't something you should use python for,
>> as it has relatively poor performance characteristics compared to some other
>> scripting languages - it's great for a lot of things, but performance isn't
>> one of them), I tend to advocate the use of shared_ptr (or some similar
>> memory management construct) over raw pointers for long-term storage anyway.
>> It can save you from a lot of headaches and opens up a lot of possibilities
>> when you combine it with weak_ptr.
>>
>> ...
>>
>> Of course, one other option that doesn't require shared pointers is to set
>> it up something like this:
>>
>> struct Element
>> {
>>     Element(const Container * c, PyObject * obj) : _container(c),
>> containerPyObj(obj)
>>     {
>>         Py_INCREF(containerPyObj);
>>     }
>>     ~Element()
>>     {
>>         Py_DECREF(containerPyObj);
>>     }
>>     std::string name() const;
>>     const Container * _container;
>>     PyObject * containerPyObj;
>> };
>>
>> Element GetElementProxy(boost::python::object self)
>> {
>>     Container * container = boost::python::extract<Container *>(self);
>>     return Element(container, self.ptr());
>> }
>>
>> BOOST_PYTHON_MODULE(libkatya)
>> {
>>     class_<Element>("Element", no_init)
>>         .def("name", &Element::name)
>>         ;
>>
>>     class_<Container>("Container", init<std::string>())
>>         .def("getElement", &GetElementProxy) // NOTE the change here, it's
>> calling the free function, not the member
>>         ;
>> }
>>
>>
>> Caveat, I haven't tested this code, so I don't know if it compiles, but
>> I've done similar things before. If you're dead set on not using shared
>> pointers for whatever reason, this should do what you want.
>>
>>
>> On 5/19/2014 1:27 AM, laan wrote:
>>>
>>> Hi,
>>>
>>> I'm having issues with Boost::Python objects that contain a reference to
>>> a
>>> c++ object:
>>>
>>> --- C++ code ----
>>>
>>> #include <boost/python.hpp>
>>>
>>> struct Container;
>>>
>>> struct Element
>>> {
>>>      Element(const Container * c) : _container(c) { }
>>>      std::string name() const;
>>>      const Container * _container;
>>> };
>>>
>>> struct Container
>>> {
>>>      Container(const std::string & s) : _string(s) { }
>>>          Element getElement()
>>>      {
>>>          return Element(this);
>>>      }
>>>      std::string name() const
>>>      {
>>>          return this->_string;
>>>      }
>>>      std::string _string;
>>> };
>>>
>>> std::string Element::name() const
>>> {
>>>      return _container->name();
>>> }
>>>
>>> using namespace boost::python;
>>>
>>> BOOST_PYTHON_MODULE(libkatya)
>>> {
>>>      class_<Element>("Element", no_init)
>>>          .def("name", &Element::name)
>>>          ;
>>>
>>>      class_<Container>("Container", init<std::string>())
>>>          .def("getElement", &Container::getElement)
>>>          ;
>>> }
>>> --- Python Code ----------------------
>>> container = test.Container("X")
>>> elem = container.getElement()
>>> del(container)
>>> print(elem.name())
>>>
>>> When calling elem.name(), it will reference the "container" object, which
>>> has been deleted, and I'm getting a segfault.
>>>
>>> How can I make boost::python increment the reference counter of Container
>>> when an element containing a pointer to the Container is returned?
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://boost.2283326.n4.nabble.com/Boost-Python-Reference-Counting-tp4662504.html
>>> Sent from the Python - c++-sig mailing list archive at Nabble.com.
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> https://mail.python.org/mailman/listinfo/cplusplus-sig
>>
>>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> https://mail.python.org/mailman/listinfo/cplusplus-sig
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> https://mail.python.org/mailman/listinfo/cplusplus-sig