[C++-sig] Retaining a useful weak_ptr to a wrapped C++ interface implemented in Python

[Adam Preble]

On Tue, Mar 6, 2012 at 3:47 AM, Holger Brandsmeier <brandsmeier at gmx.de>wrote:

> Adam,
>
> You probably have a function like this
>
> void addCallback(share_ptr<> arg) {
> callbacks.push_back(weak_ptr<>(arg);
> }
>
>
That's pretty much what I have happening.


>
> So, before the function exits your object `arg` exists at least in three
> places:
>  1) somewhere in C++ where it was created
>  2) in the python context
>  3) in the context of addCallback
>
>
Technically, it was declared and constructed from Python, but everything
you say is a consequence of this is consistent with what I'm seeing.  We
could get into semantics here.  If I create an object implementing a C++
interface, do we consider that created in Python or would it be regarded as
created in the C++ runtime?


> Assume you have class `Parent` and a class `Child` derived from it.
> Now you can do:
>  - create an instance of Child C++ and bring it to python as
> shared_ptr<Child>
>  - pass that instance to C++ (via shared_ptr<Child> or shared_ptr<Parent>)
>  - get it later back from C++ but as a shared_ptr<Parent>
>  - magic: you can treat that instance as a shared_ptr<Child>
> In C++ you would need to do a dynamic cast to get this functionallity,
> but because that object has been known to python to be an instance of
> Child, boost::python automatically makes it an instance of Child, nice
> right?
>
> Unfortunately your (and my) problem are a consequence of this. When
> you go from 2->3 boost::python prepares for doing its magic. It
> doesn't just return a copy of the shared_ptr from 1), it creates a new
> shared_ptr with a special Deallocator object. The use_count at that
> moment is 2: one for python 2) and one for addCallback() 3). When the
> function addCallback() finishes, the use_count=1 (from python) and
> weak_count=1 from 3). Once the python context ends then use_count=0
> and weak_count=1, and I believe that is exactly what you observe.
>
> In this case as use_count drops to 0 the boost custom Deallocator gets
> called. This is usally not bad, as he just deregisters (decreases the
> use cound by 1) in the shared_ptr for context 1.Only if that use_count
> in context 1) would drop to 0 the object would get deleted (that's why
> you don't observe it to be deleted). The problem is now, that the two
> weak/shared ptrs (which still point to a healthy and alive object) are
> now disconnected. So when you try to turn the weak_ptr in context3 to
> a strong pointer you would get serious problems.
>
>
Sounds about like what I'm dealing with here.  Perhaps of particular note
is that the pointers I'm moving around are typed for a parent class, but
the actual reference is to a child.


> I hope this is a correct explanation of the sitation. To solve this
> would need to revist the magic for shared_ptrs in boost::python. I
> plan to try and solve it some point later, but I am no regular
> developer for boost::python and I can not promise that I will succeed,
> nor when.
>
>
Originally I was using shared_ptr instead of weak_ptr for the callback
managers, and found some stuff never got deleted.  Could this process also
cause the disconnect there?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20120306/49b0a024/attachment.html>