[C++-sig] Properly copying wrapped classes

[Jim Bosch]

It looks like the only way to get a wrapper object is by actually 
invoking the Python constructor.  That's not what I expected, but it's 
reasonable and not completely unexpected.  And it avoids problems that 
might arise if you had any Python-only attributes attached to the 
wrapper object.

So I think you'll need to try a new approach, one that involves calling 
a wrapped copy constructor.

Would it meet your needs to just remove the __copy__ and __deepcopy__ 
implementations, and change the Python derived class to something like
what's below?

class Der(BaseClass):
     def Test(self):
         print "Python Side Test. ID: " + str(self.GetID())

def Clone(self):
     print "Python Clone"
     result = Der(...)    # whatever constructor args you need in Python
     BaseClass.__init__(result, self)
     return result


(btw, apologies to the rest of the list - I accidentally replied without 
including the list a few emails back, but all you've missed are a number 
of ideas that don't work)


Jim





On 06/08/2012 08:40 PM, Jay Riley wrote:
> I replaced my template copy with
>
> object bc_copy(object copyable, dict memo)
> {
> object copyMod = import("copy");
> object deepcopy = copyMod.attr("deepcopy");
>
> BaseClassWrap *newCopyable(new BaseClassWrap(extract<const BaseClassWrap
> &>(copyable)));
>
> BaseClass tmp = boost::python::extract<BaseClass const &>(copyable);
> object result(tmp);
>
> int copyableId = (int)(copyable.ptr());
> memo[copyableId] = result;
>
> extract<dict>(result.attr("__dict__"))().update(
> deepcopy(extract<dict>(copyable.attr("__dict__"))(),
> memo));
>
> return result;
> }
>
> and wrapped
>
> class_<BaseClass, boost::shared_ptr<BaseClassWrap> >("BaseClass", init<>())
> .def(init<const BaseClass&>())
> .def(init<const BaseClassWrap&>())
> .def("GetID", &BaseClassWrap::GetIDDefault)
> .def("Clone", &BaseClassWrap::CloneDefault)
> .def("Test", &BaseClassWrap::TestDefault)
> .def("__copy__", &generic__copy__< BaseClass >)
> .def("__deepcopy__", &bc_copy)
> ;
>
> But I'm still getting the non overriden behaviour. Calling Test on the
> clone prints out the C++ side code, while the non clone prints out the
> python side code.
>
>
>
>
>
>  > Date: Thu, 7 Jun 2012 17:13:55 -0400
>  > From: jbosch at astro.princeton.edu
>  > To: super24bitsound at hotmail.com
>  > Subject: Re: [C++-sig] Properly copying wrapped classes
>  >
>  > On 06/05/2012 12:41 AM, Jay Riley wrote:
>  > > That got rid of the compile time errors. However, On the python side I
>  > > get the following error when I try to clone:
>  > >
>  > >
>  > > File ... line 9 in Clone
>  > > return copy.deepcopy(self)
>  > > File C:\Python27\Lib\copy.py" line 174 in deepcopy
>  > > y = copier(memo)
>  > > TypeError: No to_python (by-value) converter for C++ type: class
>  > > BaseClassWrap
>  > >
>  >
>  > Hmm. I would have expected that to work. As long as BaseClass can be
>  > instantiated itself (i.e. it isn't pure abstract), you could try this in
>  > the copy/deepcopy implementation:
>  >
>  > BaseClass tmp = boost::python::extract<BaseClass const &>(copyable);
>  > object result(tmp);
>  >
>  > (I don't remember which of these types your Copyable template parameter
>  > was, but I'd recommend just trying to write this in a non-template
>  > function until you get it working).
>  >
>  > That will involve one extra copy, but it should invoke the BaseClass
>  > by-value converter instead of the BaseClassWrap by-value converter, and
>  > I *think* that should do the right thing.
>  >
>  >
>  > Jim
>  >
>  >
>  >
>  >
>  > > In the interest of trying it out, I added the BaseClassWrap to the
>  > > python module
>  > >
>  > > class_<BaseClassWrap, bases<BaseClass> >("BaseClassWrap",
> init<PyObject*>())
>  > > .def(init<PyObject*, const BaseClass&>())
>  > > .def(init<PyObject*, const BaseClassWrap&>())
>  > > .def("__copy__", &generic__copy__< BaseClassWrap >)
>  > > .def("__deepcopy__", &generic__deepcopy__< BaseClassWrap >)
>  > > ;
>  > >
>  > > But this put me back at square one, where self isn't indicative of the
>  > > cloned instance.
>  > >
>  > > Just in the interest of seeing the effect, I changed my python
> exposing to:
>  > >
>  > > class_<BaseClass, boost::shared_ptr<BaseClassWrap> >("BaseClass",
> init<>())
>  > > .def(init<const BaseClass&>())
>  > > .def(init<const BaseClassWrap&>())
>  > > .def("GetID", &BaseClassWrap::GetIDDefault)
>  > > .def("Clone", &BaseClassWrap::CloneDefault)
>  > > .def("Test", &BaseClassWrap::TestDefault)
>  > > .def("__copy__", &generic__copy__< BaseClass >)
>  > > .def("__deepcopy__", &generic__deepcopy__< BaseClass >)
>  > > ;
>  > >
>  > > But as expected, the overloading doesn't work on clones made with this
>  > > exposing. Anything else I should try?
>  > >
>  > > Again, thanks for the help so far, appreciate it.
>  > >
>  > >
>  > >
>  > >
>  > > > Date: Mon, 4 Jun 2012 11:52:42 -0400
>  > > > From: jbosch at astro.princeton.edu
>  > > > To: super24bitsound at hotmail.com
>  > > > Subject: Re: [C++-sig] Properly copying wrapped classes
>  > > >
>  > > > On 06/04/2012 12:58 AM, Jay Riley wrote:
>  > > > >
>  > > > > Hi Jim,
>  > > > > Thanks for the help so far. From what you said, I should be able to
>  > > change my deepcopy to this correct?
>  > > > >
>  > > > > template<class Copyable> object generic__deepcopy__(object
>  > > copyable, dict memo) { object copyMod = import("copy"); object deepcopy
>  > > = copyMod.attr("deepcopy");
>  > > > > Copyable *newCopyable(new Copyable(extract<const Copyable
>  > > &>(copyable))); //object
>  > >
> result(boost::python::detail::new_reference(managingPyObject(newCopyable)));
>  > > > > object result(extract<const Copyable&>(copyable)());
>  > > > > // HACK: copyableId shall be the same as the result of id(copyable)
>  > > //in Python - // please tell me that there is a better way! (and which
>  > > ;-p) int copyableId = (int)(copyable.ptr()); memo[copyableId] = result;
>  > > > > extract<dict>(result.attr("__dict__"))().update(
>  > > deepcopy(extract<dict>(copyable.attr("__dict__"))(), memo));
>  > > > > return result; }
>  > > > > When I attempt to compile it gives me the following error:
>  > > > > Error 48 error C2664: 'boost::python::api::object::object(const
>  > > boost::python::api::object&)' : cannot convert parameter 1 from
>  > > 'boost::python::api::object (__cdecl *)(boost::python::extract<T>
>  > > (__cdecl *)(void))' to 'const boost::python::api::object&'
>  > > c:\khmp\src\engine\scripting\python\scripthelpers.h 67
>  > > > > Did you mean something else? This level of boost python is above my
>  > > head, so I'm a bit lost
>  > > > >
>  > > >
>  > > > It looks like the version I gave you is a bad corner case where the
>  > > > compiler is incapable of parsing a constructor properly when
> templates
>  > > > and overloaded function-call operators are involved; I always forget
>  > > > exactly when that comes into play (not a compiler bug, just a
> weird part
>  > > > of the standard, I think). In any case, I think separating it
> into two
>  > > > lines should fix the problem:
>  > > >
>  > > > const Copyable & tmp = extract<const Copyable &>(copyable);
>  > > > object result(tmp);
>  > > >
>  > > > Let me know if that doesn't work either; at that point I should
> try to
>  > > > compile it myself and see what's going on.
>  > > >
>  > > > Jim
>  > > >
>  > > >
>  > > >
>  > > >
>  > > >
>  > > > >> Date: Sun, 3 Jun 2012 11:25:38 -0400
>  > > > >> From: jbosch at astro.princeton.edu
>  > > > >> To: cplusplus-sig at python.org
>  > > > >> CC: super24bitsound at hotmail.com
>  > > > >> Subject: Re: [C++-sig] Properly copying wrapped classes
>  > > > >>
>  > > > >> On 05/31/2012 08:49 PM, Jay Riley wrote:
>  > > > >>> I'm having a problem with some python objects derived in python.
>  > > I have
>  > > > >>> a Clone method that's supposed to make a full copy of these
> python
>  > > > >>> inherited objects, but it isn't working. The problem is my
> wrapper
>  > > > >>> class' PyObject* self isn't getting copied, so when I use
>  > > > >>> call_method<>(self, "method name"), self will never point to the
>  > > copied
>  > > > >>> instance. Here's a brief example showing the issue
>  > > > >>>
>  > > > >>
>  > > > >> <snip>
>  > > > >>
>  > > > >>>
>  > > > >>> Sorry about the length, just want to make sure I got everything
>  > > relevant
>  > > > >>> in. When I run this code, it prints:
>  > > > >>>
>  > > > >>> Python Clone
>  > > > >>> Python Side Test. ID: 1
>  > > > >>> Python Side Test. ID: 1
>  > > > >>> Original ID 1
>  > > > >>> Clone ID 1
>  > > > >>>
>  > > > >>> this is wrong because Clone ID should be 2 (inspecting the object
>  > > > >>> confirms it is 2). Like I said I'm pretty sure the problem is
>  > > that the
>  > > > >>> wrapper class' copy constructor only makes a copy of the
>  > > PyObject* self,
>  > > > >>> not a full copy. This is how every reference doc I saw was doing
>  > > class
>  > > > >>> wrapping, but isn't this wrong? Should we be making a full copy
>  > > of the
>  > > > >>> PyObject*? How would I go about doing that, or otherwise
> modifying my
>  > > > >>> classes to get the behaviour I expect.
>  > > > >>>
>  > > > >>
>  > > > >> There are a couple of things here that probably need to be
> addressed:
>  > > > >>
>  > > > >> - Be wary of relying on copy-constructor side effects to
> demonstrate
>  > > > >> whether something is working; the compiler is allowed to elide
> copy
>  > > > >> constructors under many conditions, which might result in
> misleading
>  > > > >> results. That said, I don't think that's what's causing your
> problem
>  > > > >> here (or at least not all of it).
>  > > > >>
>  > > > >> - Your implementations of __copy__ and __deepcopy__ won't work
> with
>  > > > >> polymorphic wrapper classes, because they construct a BaseClass
>  > > instance
>  > > > >> and tell Boost.Python to wrap it by managing the new
> reference. That
>  > > > >> necessarily makes a Python object that holds a BaseClass object,
>  > > not one
>  > > > >> that holds a BaseClassWrap object. If you want to get a
> BaseClassWrap
>  > > > >> Python object, you want to pass it by value to Boost.Python; that
>  > > should
>  > > > >> invoke the BaseClassWrap copy constructor and pass it a new
>  > > PyObject* to
>  > > > >> manage (so it doesn't have to copy the PyObject* itself).
>  > > > >>
>  > > > >> I think that just means you can remove the "managingPyObject"
>  > > function,
>  > > > >> and instead use:
>  > > > >>
>  > > > >> object result(extract<const Copyable&>(copyable)());
>  > > > >>
>  > > > >>
>  > > > >> HTH
>  > > > >>
>  > > > >> Jim
>  > > > >
>  > > >
>  >