[C++-sig] shared_ptr trouble

[David Abrahams]

"Ralf W. Grosse-Kunstleve" <rwgk at yahoo.com> writes:

Cc:'ing Peter Dimov in case he has any brilliant ideas.  Peter, I'll
forward you Ralf's complete posting under separate cover.

> I'm having terrible trouble understanding how Boost.Python interacts with
> shared_ptr. I'm trying to implement a small tree node class with parent-child
> relationships via weak_ptr:
>
>   class node : boost::noncopyable
>   {
>     private:
>       node() {}
>
>       boost::weak_ptr<node> parent_;
>
>     public:
>       std::string id;
>
>     // ...
>   };
>
> To be used eventually like this:
>
>   node1 = node.create(id="a")
>   node2 = node.create(id="b", parent=node1)
>
> node1 has no parent, node2 should use node1 as the parent. The equivalent C++
> code is:
>
>   std::string
>   exercise_child_parent_id()
>   { 
>     boost::shared_ptr<node> node1 = node::create("a");
>     boost::shared_ptr<node> node2 = node::create("b", node1);
>     return node2->get_parent_id();
>   }
>
> A minimal, self-contained example is attached. The test is simply:
>
> from boost_adaptbx_tree_ext import node, exercise_child_parent_id

Here's the problem:

> def exercise():
>   print "C++:", exercise_child_parent_id()
>   node1 = node.create(id="a")
>   node2 = node.create(id="b", parent=node1)

  right here, node1 gets converted to a shared_ptr<node>.  Even though
  nodes are held by shared_ptr, Boost.Python does not extract the
  held_type instance from its Python wrapper.  Instead, it creates a
  new shared_ptr (with a whole new "control block" containing a
  use_count, weak_count and deleter) whose deleter manages a reference
  count on the owning Python object.  That's the magic that allows
  Boost.Python to recover the original Python object when such a
  shared_ptr is converted back to Python.  Unfortunately, it's also
  what causes your problem.  The weak_ptr stored in your node is
  related to the control block of the new shared_ptr and _not_ to the
  one stored inside your Python object.  When the Python create( ... )
  method returns, that shared_ptr goes away.  Since it was the last
  non-weak reference to that control block, the weak_ptr thinks the
  object is gone...

>   print "Python:", node2.get_parent_id()
>
> if (__name__ == "__main__"):
>   exercise()
>
> Output:
>
> C++: a
> Python: no parent
>
> The C++ code produces the desired result, but the Python version fails. By
> adding print statements to this function of the node class
>
>       static
>       boost::shared_ptr<node>
>       create(
>         std::string const& id,
>         boost::shared_ptr<node> const& parent)

You can force Boost.Python to grab the shared_ptr stored in the Python
object by removing "const" from the line above.  That signals that a
shared_ptr lvalue is needed and no shared_ptr conjured up on the fly
by the conversion machinery will suffice (this is essentially the rule
temporaries don't bind to non-const references, as implemented by
Boost.Python).  On the other hand, when you convert that weak_ptr back
into a shared_ptr, and convert _that_ back to Python, you won't see
the original Python object, but an identical one whose held shared_ptr
refers to the same C++ object.  This is a knotty problem and I'm not
sure there's a good answer.

>       {
>         boost::shared_ptr<node> result(new node);
>         result->id = id;
>         result->parent_ = parent;
>         return result;
>       }
>
> I found out that result->parent_->id is in fact "a" right before the return,
> but back in Python the parent is lost. Since node is noncopyable I cannot
> figure out how this can be, and what I need to do to fix the problem. I'd be
> grateful for advice!

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com