[C++-sig] Efficient string passing

[Stefan Seefeld]

On 11/19/2009 07:30 AM, Mohan Ganesalingam wrote:
> Dear all,
>
> I recently sent an e-mail which (I think) didn't reach the list, about 
> trying to return large strings from C++ into Python without copying them.

I believe this is impossible: Python treats strings as immutable, which 
means you can't manipulate them in-place, and there is no C API function 
that allows you to pass in a string by reference, as that would make 
memory management impossibly complex.

> By looking at the C/Python API I've subsequently figured out roughly 
> how to do this... something like
>
> Class World
> {
>     object return_object()
>     {
>         PyObject *result = PyString_FromStringAndSize(0, 7);

This creates a new string, copying whatever you pass as input.

>         // initialise the underlying string using PyString_AsString 
>         handle<> h(result);
>         return object();

I'm not sure what your intent is, but returning a default-constructed 
object is the C++ equivalent of None.

>     }
>
> ...
>
>
> main_namespace["World"] = class_<World>("World")
> .def("return_object", &World::return_object);
>
> Unfortunately this doesn't work; when I call w.return_object(), I get 
> back None.

...because above you "return object();", presumably.

> Returning a raw PyObject * instead of a boost::python::object works, 
> but I can't tell if that involves a copy... and using 
> return_value_policy<manage_new_object>() with a PyObject * return 
> crashes.
>
> Any advice here would be very much appreciated!

I don't think you can get what you want using strings directly. You may 
create and expose a wrapper type that provides a string-like interface, 
with appropriate conversions.

     Stefan

-- 

       ...ich hab' noch einen Koffer in Berlin...