[C++-sig] to_python_converters with internal-ref-like management

[Jonathan Brandmeyer]

On Thu, 2004-09-09 at 18:04, esatel wrote:

> However, to be safe, the array_holder (which contains the multi_array
> managing the data) to last as long as the subarray. Is there a slick way to
> do this? I realize that I can eliminate the need for this dependence through
> copies, but the size of the arrays involved (slices >10000 doubles)
> motivates me to avoid the wasted motion.
> 
> Thanks,
> Eli

Try this:

-Manage the array_holder's memory with shared_ptr, or some other
refcounting system.
-Create a dummy class for Python which does nothing other than own a
reference to the underlying array_holder.
-Convert the subarray to a _numarray_-based Python array (because this
trick won't work with with Numeric).
-Make a boost::python::object from the PyObject* you just initialized
(I'll call it 'ret' for later reference).  Make sure you do it in a way
that doesn't delete the underlying PyObject too early, or add an extra
reference to it.
-Call ret.attr( "some_unique_name") = object(dummy(array_holder));
-Return ret to Python.

Here's how I think it should work.  Because numarray uses the new class
system, you can add arbitrary attributes to numarray objects.  So, you
add an attribute that holds a reference to the underlying data.  When
the refcount on the PyArray goes to zero, so will the extra attribute's
refcount (assuming that someone else didn't reference it again somehow -
that is why you use the hard-to-guess name*).  When Python destroys the
dummy attribute, the refcount on the data to which that slice object
pointed gets its refcount decremented.

I haven't tried this before, so if you try it, and it works, please let
folks know.

HTH,
-Jonathan

*I would even go as far as salting the name with a wee bit of random
data, but that might be overkill.