[C++-sig] Boost.Python: Caching Python object instances

[Doug Gregor]

Hello,

I am using Boost.Python to expose the Boost Graph Library in Python.  
The vertices in the graph types are small structures that are always  
passed by value. A vertex looks something like this:

	struct Vertex {
	  int index;
	  Graph* graph;
	};

This class is exposed to python in the normal way:

	class_<vertex_descriptor>();

Now, each time that we convert a Vertex from C++ to Python,  
Boost.Python will create a new PyObject for it. This behavior has  
surprised us a few times, and contributes to some performance  
problems. So, I would like to cache the PyObject instances in the  
Graph object itself:

	class Graph {
	public:
	  std::vector<boost::python::object> py_vertices;
	};

When I add a new vertex, I want to create a Python object for it and  
store that object in the py_vertices vector. Then, when Boost.Python  
needs to convert a vertex_descriptor value "v" to a Python object,  
I'd like to return "py_vertices[v.index]" instead of creating a new  
object.

With lots of hackery, I've attained some limited success in this area.

My approach is to register a to_python_vertex for the vertex type  
with this function object:

   struct cached_vertex_object {
     static PyObject* convert(const Vertex& v) {
         PyObject* result = v.graph->py_vertices[v.index].ptr();
         Py_INCREF(result);
         return result;
     }
   };

   to_python_converter<Vertex, cached_vertex_object>();

Of course, now I have the problem of creating the  
boost::python::object values to put in the cache! Digging around in  
the source a bit, I came up with this:

   typedef class_cref_wrapper<Vertex,
                              make_instance<Vertex,  
value_holder<Vertex> > >
     Converter;

   Vertex result(this->py_vertices.size(), this);
   boost::python::object vertex_object(handle<>(Converter::convert 
(result)));
   this->py_vertices.push_back(result);

This actually does work, but I get a sharp rebuke from Boost.Python  
whenever my extension module gets loaded:

/u/dgregor/lib/python/boost/graph/__init__.py:10: RuntimeWarning: to- 
Python converter for boost::graph::python::basic_descriptor<void*,  
boost::undirectedS> already registered; second conversion method  
ignored.
   from _graph import *

I have two questions:

	1) Is there a better way to cache Python object instances with  
Boost.Python?

	2) I've filtered out those Boost.Python warnings: will they come  
back to bite me eventually?

	Cheers,
	Doug