[C++-sig] [Boost.Python] How to adjust Python reference counts for hybrid objects?

[Per Knudsgaard]

   Thanks for your answer.

   I generally prefer intrusive pointers for a number of reasons:  They are safer (no problems converting between raw and managed).  They are generally faster (no cache misses because the reference count is embedded in your object).  They don't require extra allocations (saving a few bytes for every object).  None of those are really that significant here, but since I have no libraries or third party objects in my application, I don't encounter the most obvious downside.

   That said, after writing the email last night, I realized that I could make a minor change to PtrBase which eliminates the need for the share function.  It has a slight performance cost (one indirection but it is also in the pure C++ case which bugs me a little) and it made the code much cleaner.  The strange thing is that I no longer need to call the Py_INCREF function twice.  Ah, well.

   I have mostly used SWIG in the past so I am on a learning curve with Boost.Python.  Apart from somewhat cryptic error messages, I like it.  At some point I will start digging in the code to see how the details are working.  Maybe I can make some charts as I go along :)

   -- Per.

-----Original Message-----
From: Jim Bosch [mailto:talljimbo at gmail.com] 
Sent: Tuesday, November 15, 2011 3:43 PM
To: Development of Python/C++ integration
Cc: Per Knudsgaard
Subject: Re: [C++-sig] [Boost.Python] How to adjust Python reference counts for hybrid objects?

On 11/14/2011 09:54 PM, Per Knudsgaard wrote:
> I have gotten (3) to work, but I am wondering about two things.
> Since I couldn't get the constructor to work, I went for a modifier 
> that returns the object.

Sorry no one responded to your questions earlier. I havne't looked too closely yet, but I thought I'd point out that you have basically written a big workaround for the fact that you're using intrusive_ptr instead of shared_ptr. Pretty much all of the things you're trying to do work out-of-the-box with shared_ptr, and you don't have to worry about the reference count yourself at all.

Unfortunately, similar support for intrusive_ptr isn't really present in Boost.Python, but that's because intrusive_ptr isn't nearly as flexible.

>
> So, two questions:
>
> *         Is this a legal/recommended way to do a modifier?

Legal, yes.  I think.  If switching to shared_ptr is a possibility, I'd consider that the recommended way to deal with the entire problem. 
Without it, I think you're in rather uncharted territory.

> *         If I only do a single INCREF, then the object is destroyed
> while executing the NewObject().share() call.  Why do I need two 
> INCREFs?

I'm a bit stumped by this, too, but I haven't looked as closely as I might have if you didn't seem to have a satisfactory solution or if I didn't think shared_ptr was really the way to go here.  Part of the explanation might be the fact that Boost.Python doesn't hold a reference to what you get out of get_owner(), so you don't have one either, and that might do funny things when the only Python reference is a temporary object you're calling the share() method on.  But I still would have expected that to be safe.

HTH

Jim