From uschmitt at mineway.de Tue May 3 17:07:57 2011 From: uschmitt at mineway.de (Uwe) Date: Tue, 03 May 2011 17:07:57 +0200 Subject: [C++-sig] Fwd: [personal] boost-python: linking external libries results in unresolved symbols In-Reply-To: References: Message-ID: Hi, I have a problem building an python extension module with several external shared libraries. My Jamroot file looks like this: use-project boost : C:/boost_1_42_0 ; project boost-python-quickstart : requirements /boost/python//boost_python ; import python ; lib openms : : OpenMS ; lib xerces : : xerces-c_3_0 ; lib qtcore : : QtCore4 ; python-extension pyms : wrapper.cpp : ../OpenMS/include/ C:/Qt/4.7.2/include ../OpenMS/contrib/src/xerces-c-r806068/src/ C:/Qt/4.7.2/include/QtCore/ ../OpenMS/contrib/include/ openms xerces qtcore ; install convenient_copy : pyms : on SHARED_LIB PYTHON_EXTENSION . ; Building with "bjam toolset=msvc" works fine. I'm using Visual Studio 10 Express. But loading of the Python module fails due to unresolved symbols. When I use Dependency Walker for investigating the generated module, I get the information, that QtCore4 and xerxes libs have an unresolved "Parent import" of __Py_NoneStruct. Can anybody help me ? My knowledge about Windows internals is quite limited. Regards, Uwe -- Dr. rer. nat. Uwe Schmitt Forschung & Entwicklung Mathematik mineway GmbH Geb?ude 4 Im Helmerswald 2 66121 Saarbr?cken Telefon: +49 (0)681 8390 5334 Telefax: +49 (0)681 830 4376 uschmitt at mineway.de www.mineway.de Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer Amtsgericht Saarbr?cken HRB 12339 -- Dr. rer. nat. Uwe Schmitt Forschung & Entwicklung Mathematik mineway GmbH Geb?ude 4 Im Helmerswald 2 66121 Saarbr?cken Telefon: +49 (0)681 8390 5334 Telefax: +49 (0)681 830 4376 uschmitt at mineway.de www.mineway.de Gesch?ftsf?hrung: Dr.-Ing. Mathias Bauer Amtsgericht Saarbr?cken HRB 12339 -------------- next part -------------- An HTML attachment was scrubbed... URL: From zebbey at gmail.com Wed May 4 17:01:07 2011 From: zebbey at gmail.com (zeb) Date: Wed, 4 May 2011 08:01:07 -0700 (PDT) Subject: [C++-sig] transfer ownership problem Message-ID: <1304521267333-3495846.post@n4.nabble.com> hi, I have some problem about transfer-ownership. C++ code is like this: class RefCounter { int _count; public: RefCounter() : _count(1) {} virtual RefCounter() {} void addRef() { ++_count; } void release() { --_count; if (_count == 0) delete this; } }; class A : public RefCounter {...}; class B { public: void setA(A* a) { // B takes the ownership of A. _a = a; a.addRef(); } private: A* _a; }; So, in python, I use these classes like: a = A() # the reference count of a is 1 now. b = B() b.setA(a) # B take the ownership of A, and now the reference count is 2. a.release() # a should call release to decrease the reference count, but now a is invalid. Python says: did not match C++ signature. I can change the C++ code to solve this problem, but I really don't want to change C++ code. It's better to solve it by other way. Is there any one could give me some advice? THANK YOU VERY MUCH. -- View this message in context: http://boost.2283326.n4.nabble.com/transfer-ownership-problem-tp3495846p3495846.html Sent from the Python - c++-sig mailing list archive at Nabble.com. From talljimbo at gmail.com Thu May 5 04:28:16 2011 From: talljimbo at gmail.com (Jim Bosch) Date: Wed, 04 May 2011 19:28:16 -0700 Subject: [C++-sig] transfer ownership problem In-Reply-To: <1304521267333-3495846.post@n4.nabble.com> References: <1304521267333-3495846.post@n4.nabble.com> Message-ID: <4DC20B40.2070908@gmail.com> On 05/04/2011 08:01 AM, zeb wrote: > hi, I have some problem about transfer-ownership. C++ code is like this: > > class RefCounter { > int _count; > public: > RefCounter() : _count(1) {} > virtual RefCounter() {} > void addRef() { ++_count; } > void release() { --_count; if (_count == 0) delete this; } > }; > > class A : public RefCounter {...}; > > class B { > public: > void setA(A* a) { // B takes the ownership of A. > _a = a; > a.addRef(); > } > private: > A* _a; > }; > > So, in python, I use these classes like: > a = A() # the reference count of a is 1 now. > b = B() > b.setA(a) # B take the ownership of A, and now the reference count is 2. > a.release() # a should call release to decrease the reference count, but now > a is invalid. Python says: did not match C++ signature. > > I can change the C++ code to solve this problem, but I really don't want to > change C++ code. > It's better to solve it by other way. > > Is there any one could give me some advice? You should really to let Python (and C++, for that matter) handle your reference counting if at all possible. The best way to do that would be to ensure your Python objects are held inside a smart pointer (boost::intrusive_ptr would probably work). If you set that up correctly, you should never have to call addRef() or release() from your Python code - think of it as allowing Python to always own one reference, which it is responsible for deleting when its done with it. I realize I'm being a little vague about the details - feel free to ask more questions if you need more help, but a look through the Boost.Python reference documentation for setting the "HeldType" of a class_ instantiation to a smart pointer should get you started. Good luck! Jim Bosch > > -- > View this message in context: http://boost.2283326.n4.nabble.com/transfer-ownership-problem-tp3495846p3495846.html > Sent from the Python - c++-sig mailing list archive at Nabble.com. > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig From zebbey at gmail.com Thu May 5 10:16:40 2011 From: zebbey at gmail.com (zeb) Date: Thu, 5 May 2011 01:16:40 -0700 (PDT) Subject: [C++-sig] transfer ownership problem In-Reply-To: <4DC20B40.2070908@gmail.com> References: <1304521267333-3495846.post@n4.nabble.com> <4DC20B40.2070908@gmail.com> Message-ID: <1304583400888-3497818.post@n4.nabble.com> Thank you, Jim. ^-^ By using boost::intrusive_ptr, the problem is solved. Everything works well. -- View this message in context: http://boost.2283326.n4.nabble.com/transfer-ownership-problem-tp3495846p3497818.html Sent from the Python - c++-sig mailing list archive at Nabble.com. From zebbey at gmail.com Sat May 7 03:51:34 2011 From: zebbey at gmail.com (zeb) Date: Fri, 6 May 2011 18:51:34 -0700 (PDT) Subject: [C++-sig] how to call base class method? Message-ID: <1304733094439-3504749.post@n4.nabble.com> I have 2 class exposed to python: class A { public: virtual int get() { return 1; } }; class B : public A { public: virtual int get() { return 2; } }; In python: obj = B() obj.get() # this will call B's get() For some reason, I want to call A's get() in python, like obj->A::get() in C++. So I have tried: @ Directly call base class's method in python: A.get(obj) # failed. @ make a wrap function to cast B to A: A* cast(B* obj) { return (A*)obj; } And try to use different call policy, like manage_new_object, reference_existing_object, return_internal_reference, but all failed. After casting, the two object actually the same object. >>> objB = B() >>> objA = cast(objB) >>> objA == objB True Please help me THANK YOU -- View this message in context: http://boost.2283326.n4.nabble.com/how-to-call-base-class-method-tp3504749p3504749.html Sent from the Python - c++-sig mailing list archive at Nabble.com. From talljimbo at gmail.com Sat May 7 04:24:41 2011 From: talljimbo at gmail.com (Jim Bosch) Date: Fri, 06 May 2011 19:24:41 -0700 Subject: [C++-sig] how to call base class method? In-Reply-To: <1304733094439-3504749.post@n4.nabble.com> References: <1304733094439-3504749.post@n4.nabble.com> Message-ID: <4DC4AD69.2090102@gmail.com> On 05/06/2011 06:51 PM, zeb wrote: > I have 2 class exposed to python: > class A { > public: > virtual int get() { return 1; } > }; > > class B : public A { > public: > virtual int get() { return 2; } > }; > > In python: > obj = B() > obj.get() # this will call B's get() > > For some reason, I want to call A's get() in python, like obj->A::get() in > C++. So I have tried: > @ Directly call base class's method in python: A.get(obj) # failed. > @ make a wrap function to cast B to A: > A* cast(B* obj) { > return (A*)obj; > } > And try to use different call policy, like manage_new_object, > reference_existing_object, return_internal_reference, but all failed. > After casting, the two object actually the same object. >>>> objB = B() >>>> objA = cast(objB) >>>> objA == objB > True > The fact that you never get back an "A" Python object is very much a feature rather than a bug, and while you might be able to find a way around it, it's not recommended. If this wasn't a virtual member function, you could have used the usual Python syntax for a base class method call: obj = B() A.get(obj) # should return 1, if "get" isn't virtual For a virtual member function, I don't think there's a way to do this without wrapping the base class method separately, and even that requires some extra work. I think something like this may do what you want: int A_get(A & self) { return self.A::get(); } class_("A") .def("A_get", &A_get) .def("get", &A::get) ; You could also wrap "A_get" directly as "get" in class A, which should allow you to use the "A.get(obj)" successfully, I think, but you might want to make sure that it doesn't also destroy the usual (virtual) behavior of the method in Python. Jim From zebbey at gmail.com Sat May 7 07:55:13 2011 From: zebbey at gmail.com (zeb) Date: Fri, 6 May 2011 22:55:13 -0700 (PDT) Subject: [C++-sig] how to call base class method? In-Reply-To: <4DC4AD69.2090102@gmail.com> References: <1304733094439-3504749.post@n4.nabble.com> <4DC4AD69.2090102@gmail.com> Message-ID: <1304747713779-3505083.post@n4.nabble.com> Making a A_get wrapper seems to be the only way to solve this problem. THANK YOU JIM -- View this message in context: http://boost.2283326.n4.nabble.com/how-to-call-base-class-method-tp3504749p3505083.html Sent from the Python - c++-sig mailing list archive at Nabble.com. From engelbert.Tijskens at biw.kuleuven.be Mon May 9 14:56:29 2011 From: engelbert.Tijskens at biw.kuleuven.be (Engelbert Tijskens) Date: Mon, 9 May 2011 14:56:29 +0200 Subject: [C++-sig] function that modifies an existing object Message-ID: Hi, i need some help on this. //c++ class Simulation { Simulation() {/*...*/} run() {/**/} }; void modifySimulation( Simulation & sim ) {/*...*/} BOOST_PYTHON_MODULE(pySimulation) { using namespace boost::python; class_("Simulation") .def("run" ,&Simulation_t::run ) ; def("modifySimulation",modifySimulation); } #python import pySimulation sim = pySimulation.Simulation() #fine modifySimulation(sim) #fine as well, I can debug it, and it does the right thing sim.run() on this last statement the system crashes. python is basically telling me that sim is unbound. it seems to mee that the Simulation object is destroyed, but i cannot see why nor how i am supposed to prevent this using boost.python From Holger.Joukl at LBBW.de Mon May 9 17:14:38 2011 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Mon, 9 May 2011 17:14:38 +0200 Subject: [C++-sig] BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS default args core dump Message-ID: Hi, this is basically a re-post of a problem I posted 5 weeks ago, on which there's been no echo whatsoever. Now, I'm unsure if this is because I posted on April 1st, nobody has ever seen this problem on his platform, nobody ever uses the BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro this way or I'm doing s.th. blatantly stupid. What nags me is that I don't think I'm doing something exotic and, moreover, that I see different behaviour depending on execution context, i.e. run in a script with or without previous method call(s) vs interactive interpreter session. I'll try to summarize the problem a bit more(see below for a link to the original post for reference): I'm having trouble wrapping a very simple member function with Boost.Python using the BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro, getting a segmentation fault (sometimes a bus error). I run into the problem both with Boost 1.44.0 and 1.46.1, running on Solaris 10/Sparc using gcc 4.5.1 and Python 2.7.1. I can reproducibly avoid the segfault and see an (expected) exception iff the code is * not run in an interactive interpreter session and * if there is a boost-induced exception succesfully raised before the critical call (which I don't understand at all). ##### wrapped class // file default_arguments_class.hpp class DefaultArgs { public: // member functions int foo(int arg1=100, int arg2=10) { return arg1 - arg2; }; }; ##### wrapper code // file default_arguments_wrap.cpp #include #include "default_arguments_class.hpp" namespace bp = boost::python; BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(DefaultArgs_foo_overloads, DefaultArgs::foo, 0, 2) BOOST_PYTHON_MODULE(defaultargs) { bp::class_("DefaultArgs", "DefaultArgs class docstring") .def("foo_macro_a2", &DefaultArgs::foo, DefaultArgs_foo_overloads((bp::arg("arg2")))) ; }; ##### In use in interactive interpreter session: >>> import defaultargs >>> d = defaultargs.DefaultArgs() >>> >>> try: ... print "d.foo_macro_a2(1, 2, 3):", d.foo_macro_a2(1, 2, 3) ... except Exception, e: ... print e ... d.foo_macro_a2(1, 2, 3): Python argument types in DefaultArgs.foo_macro_a2(DefaultArgs, int, int, int) did not match C++ signature: foo_macro_a2(DefaultArgs {lvalue}) foo_macro_a2(DefaultArgs {lvalue}, int) foo_macro_a2(DefaultArgs {lvalue}, int, int arg2) >>> print "d.foo_macro_a2(arg2=60):", d.foo_macro_a2(arg2=60) Bus Error (core dumped) ##### In use within a script: $ cat foo.py import defaultargs d = defaultargs.DefaultArgs() try: print "d.foo_macro_a2(1, 2, 3):", d.foo_macro_a2(1, 2, 3) except Exception, e: print e print "d.foo_macro_a2(arg2=60):", d.foo_macro_a2(arg2=60) $ PYTHONPATH=/var/tmp/boost_apps/boost/build/boost_1_46_1/py2.7/minimal/gcc-4.5.1 /debug/ /apps/local/gcc/4.5.1/bin/python2.7 foo.py d.foo_macro_a2(1, 2, 3): Python argument types in DefaultArgs.foo_macro_a2(DefaultArgs, int, int, int) did not match C++ signature: foo_macro_a2(DefaultArgs {lvalue}) foo_macro_a2(DefaultArgs {lvalue}, int) foo_macro_a2(DefaultArgs {lvalue}, int, int arg2) d.foo_macro_a2(arg2=60): Traceback (most recent call last): File "/ae/data/tmp/hjoukl/foo.py", line 9, in print "d.foo_macro_a2(arg2=60):", d.foo_macro_a2(arg2=60) Boost.Python.ArgumentError: Python argument types in DefaultArgs.foo_macro_a2(DefaultArgs) did not match C++ signature: foo_macro_a2(DefaultArgs {lvalue}) foo_macro_a2(DefaultArgs {lvalue}, int) foo_macro_a2(DefaultArgs {lvalue}, int, int arg2) My original post complete with bjam etc. can be found here: http://article.gmane.org/gmane.comp.python.c%2B%2B/15163 Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From rwgk at yahoo.com Mon May 9 18:34:19 2011 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Mon, 9 May 2011 09:34:19 -0700 (PDT) Subject: [C++-sig] BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS default args core dump In-Reply-To: References: Message-ID: <594832.17022.qm@web111402.mail.gq1.yahoo.com> Hi Holger, You don't need the BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro if you support keyword arguments (which is usually best). It should be as simple as: .def("foo", &DefaultArgs::foo, (bp::arg("arg1")=100, bp::arg("arg2")=10)) The FUNCTION_OVERLOADS macros are more or less a relict. It is almost always best to follow the recipe above. General remarks: use valgrind to find sources of segmentation faults and bus errors; look in boost/libs/python/test for role models to follow (everything you see in there is certain to work). Ralf >________________________________ >From: Holger Joukl >To: cplusplus-sig at python.org >Sent: Monday, May 9, 2011 8:14 AM >Subject: Re: [C++-sig] BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADSdefault args core dump > > >Hi, > >this is basically a re-post of a problem I posted 5 weeks ago, on which >there's >been no echo whatsoever. Now, I'm unsure if this is because I posted on >April 1st, >nobody has ever seen this problem on his platform, nobody ever uses the >BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro this way or I'm doing s.th. >blatantly >stupid. > >What nags me is that I don't think I'm doing something exotic and, >moreover, that >I see different behaviour depending on execution context, i.e. run in a >script with >or without previous method call(s) vs interactive interpreter session. > >I'll try to summarize the problem a bit more(see below for a link to the >original post >for reference): > >I'm having trouble wrapping a very simple member function with Boost.Python >using the >BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro, getting a segmentation fault >(sometimes >a bus error). > >I run into the problem both with Boost 1.44.0 and 1.46.1, running on >Solaris 10/Sparc >using gcc 4.5.1 and Python 2.7.1. > >I can reproducibly avoid the segfault and see an (expected) exception iff >the code is >* not run in an interactive interpreter session and >* if there is a boost-induced exception succesfully raised before the >critical call >(which I don't understand at all). > >##### wrapped class >// file default_arguments_class.hpp >class DefaultArgs { >public: // member functions >? ? int foo(int arg1=100, int arg2=10) { return arg1 - arg2; }; >}; > >##### wrapper code > >// file default_arguments_wrap.cpp > >#include >#include "default_arguments_class.hpp" > >namespace bp = boost::python; > >BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(DefaultArgs_foo_overloads, >DefaultArgs::foo, 0, 2) >BOOST_PYTHON_MODULE(defaultargs) >{ >? ? bp::class_("DefaultArgs", "DefaultArgs class docstring") >? ? ? ? .def("foo_macro_a2", &DefaultArgs::foo, >DefaultArgs_foo_overloads((bp::arg("arg2")))) >? ? ; >}; > >##### In use in interactive interpreter session: > >>>> import defaultargs >>>> d = defaultargs.DefaultArgs() >>>> >>>> try: >...? ???print "d.foo_macro_a2(1, 2, 3):", d.foo_macro_a2(1, 2, 3) >... except Exception, e: >...? ???print e >... >d.foo_macro_a2(1, 2, 3): Python argument types in >? ? DefaultArgs.foo_macro_a2(DefaultArgs, int, int, int) >did not match C++ signature: >? ? foo_macro_a2(DefaultArgs {lvalue}) >? ? foo_macro_a2(DefaultArgs {lvalue}, int) >? ? foo_macro_a2(DefaultArgs {lvalue}, int, int arg2) >>>> print "d.foo_macro_a2(arg2=60):", d.foo_macro_a2(arg2=60) >Bus Error (core dumped) > >##### In use within a script: >$ cat foo.py >import defaultargs >d = defaultargs.DefaultArgs() > >try: >? ? print "d.foo_macro_a2(1, 2, 3):", d.foo_macro_a2(1, 2, 3) >except Exception, e: >? ? print e > >print "d.foo_macro_a2(arg2=60):", d.foo_macro_a2(arg2=60) > >$ >PYTHONPATH=/var/tmp/boost_apps/boost/build/boost_1_46_1/py2.7/minimal/gcc-4.5.1 >/debug/ /apps/local/gcc/4.5.1/bin/python2.7 foo.py >d.foo_macro_a2(1, 2, 3): Python argument types in >? ? DefaultArgs.foo_macro_a2(DefaultArgs, int, int, int) >did not match C++ signature: >? ? foo_macro_a2(DefaultArgs {lvalue}) >? ? foo_macro_a2(DefaultArgs {lvalue}, int) >? ? foo_macro_a2(DefaultArgs {lvalue}, int, int arg2) >d.foo_macro_a2(arg2=60): >Traceback (most recent call last): >? File "/ae/data/tmp/hjoukl/foo.py", line 9, in >? ? print "d.foo_macro_a2(arg2=60):", d.foo_macro_a2(arg2=60) >Boost.Python.ArgumentError: Python argument types in >? ? DefaultArgs.foo_macro_a2(DefaultArgs) >did not match C++ signature: >? ? foo_macro_a2(DefaultArgs {lvalue}) >? ? foo_macro_a2(DefaultArgs {lvalue}, int) >? ? foo_macro_a2(DefaultArgs {lvalue}, int, int arg2) > >My original post complete with bjam etc. can be found here: >http://article.gmane.org/gmane.comp.python.c%2B%2B/15163 > >Holger > > >Landesbank Baden-Wuerttemberg >Anstalt des oeffentlichen Rechts >Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz >HRA 12704 >Amtsgericht Stuttgart > >_______________________________________________ >Cplusplus-sig mailing list >Cplusplus-sig at python.org >http://mail.python.org/mailman/listinfo/cplusplus-sig > > > From Holger.Joukl at LBBW.de Wed May 11 13:13:51 2011 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Wed, 11 May 2011 13:13:51 +0200 Subject: [C++-sig] BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS default args core dump In-Reply-To: <594832.17022.qm@web111402.mail.gq1.yahoo.com> References: <594832.17022.qm@web111402.mail.gq1.yahoo.com> Message-ID: Hi Ralf, > You don't need the BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro if > you support keyword arguments (which is usually best). It should be > as simple as: > > .def("foo", &DefaultArgs::foo, (bp::arg("arg1")=100, bp::arg("arg2")=10)) > > > The FUNCTION_OVERLOADS macros are more or less a relict. It is > almost always best to follow the recipe above. I see. I had already successfully tried your proposed solution but was curious if I could use the macro mechanism for variable arg lists. > General remarks: use valgrind to find sources of segmentation faults > and bus errors; look in boost/libs/python/test for role models to > follow (everything you see in there is certain to work). Unfortunately, Solaris/Sparc is not supported by valgrind. Maybe I'll get around to try the same on Linux but for now I'll just stick to your advice. Thanks a lot Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From rwgk at yahoo.com Wed May 11 17:44:54 2011 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Wed, 11 May 2011 08:44:54 -0700 (PDT) Subject: [C++-sig] BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS default args core dump In-Reply-To: References: <594832.17022.qm@web111402.mail.gq1.yahoo.com> Message-ID: <997321.81664.qm@web111401.mail.gq1.yahoo.com> > I see. I had already successfully tried your proposed solution but was curious if > I could use the macro mechanism for variable arg lists. For the record: If you don't want/need to support keyword arguments, the macro mechanism enables you to have variable argument lists without having to specify the default values.I don't think there is a way to avoid the default value duplication (once in C++ code, then again in Boost.Python bindings) if you need keyword arguments. In practice I find the keyword arguments much more important than the drawback of having to duplicate the default values, so I never use the macro mechanism. Ralf From Rainer.Kluger at LBBW.de Thu May 12 14:57:17 2011 From: Rainer.Kluger at LBBW.de (Rainer Kluger) Date: Thu, 12 May 2011 14:57:17 +0200 Subject: [C++-sig] from_python conversion for a const char*& argument Message-ID: Hi I try to expose the following class to python (using gcc 4.5.1, boost 1_44_0, Python 2.7.1): // C++ class A { public: int getName(const char*& name); // suppose the implementation assigns the const char* x = "foo"; variable to the ref-pointer } I'd like to expose this API as is and not alter the method interface so I think I'll have to do s.th. like this: ==================================== # intended usage in Python: >>> import test >>> a = A() >>> x = const_char_ptr_ref() >>> y = a.getName(x) >>> x() 'foo' ==================================== I followed the "recipes" of the python test example, realizing that - m1.cpp -extract.cpp and tried to get the things together, knowing that those example do not really match my use case. My understanding for what I need to do, are the following steps: (1) encapsulate the const char*& variable in a container class, let's say ConstCharPtrRef"" (2) expose the ConstCharPtrRef class to Python (3) provide a factory function for the class exposed in the step (2) (4) expose the factory function from step (3) (5) provide an extractor function which extracts the "const char*&" variable from within the given python argument (6) register a from_python converter for the given type and the provided extractor function ==================================== Here is what I tried so far (Remark: This is code is not compilable/running ) ==================================== // Step 1: Container class class ConstCharPtrRef { public: ConstCharPtrRef(): c_ptr(NULL), x(c_ptr) { }; const char *c_ptr; const char *&x; };; // Step 3: Factory function object const_char_ptr_ref() { return object(boost::cref(new ConstCharPtrRef())); }; // Step 4: // Extractor 1.st Try struct extract_const_char_ptr_ref { ConstCharPtrRef& p = extract(o); return p.x; }; // Extractor 2.nd Try static const char*& execute(PyObject* o) { ConstCharPtrRef& p = extract(o); return p.x; }; // BOOST_PYTHON_MODULE(test) { // Step 2: Expose container class stuff class_("ConstCharPtrRef", no_init); // Step 4: def("const_char_ptr_ref", &const_char_ptr_ref); // Step 5: register from_python converter (Both tries are commented out) // from_python registration: 1.Try // lvalue_from_pytype< &extract_const_char_ptr_ref, type_id() >(); // causes ERROR (1) see below // from_python registration: 2.Try: // boost::python::converter::registry::insert( &execute, type_id()); // causes ERROR (2) see below // Finally: Expose the class "A" bp::class_("A") .def("getName", &A::getName) ; }; ==================================== My questions are: 1. Are the steps I mentioned above correct, or is there something missing 2. I tried different ways (see code snippet above for the names) to get the things together with no success: 1.st Try: - extractor: "struct extract_const_char_ptr_ref" with static "execute" method - registration: "lvalue_from_python()" ERROR (1) ==> this one complains: error: wrong number of template arguments (1, should be 2) boost_1_44_0/boost/python/lvalue_from_pytype.hpp:87:8: error: provided for 'template struct boost::python::lvalue_from_pytype' 2.nd Try: - extractor: global static "execute" function - registration: boost::python::converter::registry::insert () ERROR (2) ==> this one complains: error: call of overloaded 'insert(const char*& (*)(PyObject*), boost::python::type_info)' is ambiguous boost_1_44_0/boost/python/converter/registry.hpp:30:26: note: candidates are: void boost::python::converter::registry::insert(PyObject* (*)(const void*), boost::python::type_info, const PyTypeObject* (*)()) boost_1_44_0/boost/python/converter/registry.hpp:33:26: note: void boost::python::converter::registry::insert(void* (*)(PyObject*), boost::python::type_info, const PyTypeObject* (*)()) I also tried to adapt the scenario found in http://boost.2283326.n4.nabble.com/C-sig-from-python-converter-problem-unable-to-match-C-signature-td2698994.html to my case, but wasn't successful. So I really have no clue, which is the right Boost.Python to get the things done. Any help would be appreciated to get this running Thanks Rainer Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From stefan at seefeld.name Thu May 12 15:28:15 2011 From: stefan at seefeld.name (Stefan Seefeld) Date: Thu, 12 May 2011 09:28:15 -0400 Subject: [C++-sig] from_python conversion for a const char*& argument In-Reply-To: References: Message-ID: <4DCBE06F.4080906@seefeld.name> Hi Rainer, On 2011-05-12 08:57, Rainer Kluger wrote: > > Hi > > I try to expose the following class to python (using gcc 4.5.1, boost > 1_44_0, Python 2.7.1): > > > // C++ > class A > { > public: > int getName(const char*& name); // suppose the implementation > assigns the const char* x = "foo"; variable to the ref-pointer > } > > I'd like to expose this API as is and not alter the method interface so I > think I'll have to do s.th. like this: > > ==================================== > > # intended usage in Python: > >>>> import test >>>> a = A() >>>> x = const_char_ptr_ref() >>>> y = a.getName(x) >>>> x() > 'foo' > > ==================================== > > > I followed the "recipes" of the python test example, realizing that > - m1.cpp > -extract.cpp > and tried to get the things together, knowing that those example do not > really match my use case. > > My understanding for what I need to do, are the following steps: > > (1) encapsulate the const char*& variable in a container class, let's > say ConstCharPtrRef"" > (2) expose the ConstCharPtrRef class to Python > (3) provide a factory function for the class exposed in the step (2) > (4) expose the factory function from step (3) Right. > (5) provide an extractor function which extracts the "const char*&" > variable from within the given python argument > (6) register a from_python converter for the given type and the > provided extractor function I'm not sure I understand the need for (5) and (6). If you have defined a ConstCharPtrRef C++ type, and exported it to Python, you will already have converters between the two, so from within C++ you then can simply access the ConstCharPtrRef members by whatever API you define in C++. There is no need for extra converters at that point. I'm also not sure the reference needs to be captured at that level, as that's more of a call policy (for the A::getName member function) than an aspect of the type. Here is how I would approach it (caution: fully untested code): struct ConstCharPtr { char const *x; }; // A factory... ConstCharPtr make_ccp(...); // the A::getName, adjusted to work with a ConstCharPtr argument int get_name(A const &a, ConstCharPtr const &ccp) { return a.getName(ccp);} BOOST_PYTHON_MODULE(test) { // Export the ConstCharPtr type... class_ ccp("ConstCharPtr", no_init); // ...and define how to construct it ccp.def("__init__", make_cpp); // Export the A type... class_ a("A"); // and export the getName member function a.def("getName", get_name); //... } I believe this should actually do what you want. Stefan -- ...ich hab' noch einen Koffer in Berlin... From Rainer.Kluger at LBBW.de Fri May 13 11:56:21 2011 From: Rainer.Kluger at LBBW.de (Rainer Kluger) Date: Fri, 13 May 2011 11:56:21 +0200 Subject: [C++-sig] Antwort: Re: from_python conversion for a const char*& argument In-Reply-To: <4DCBE06F.4080906@seefeld.name> References: <4DCBE06F.4080906@seefeld.name> Message-ID: Hi Stefan, first of all thanks for your quick response. You put me on the right track. Your explanation clarified some C++/ Boost.Python issues I wasn't aware of before. Your sketched solution is exactly what I need. Below is the complete running example. Thanks Rainer ############################################################################## # # const_char_ptr.cpp # ############################################################################## #include #include // Container Class struct ConstCharPtr { char const *x; std::string operator()() const { std::string s(x); return s; }; }; const char* x= "A.B"; // Class 'A' class A { public: A(): m_ccp(NULL) { }; int getName(const char *& ccp) {m_ccpr = ccp; ccp = x; return 1; } std::string operator()() const { std::string s(m_ccp); return s; }; private: const char* m_ccp; }; // Container class factory ConstCharPtr make_ccp() { return ConstCharPtr(); }; // the A::getName, adjusted to work with a ConstCharPtr argument int get_name(A &a, ConstCharPtr &ccp) { return a.getName(ccp.x);} namespace bp = boost::python; // expose to Python void export_ccp() { bp::class_ ccp("ConstCharPtr"); ccp.def("__init__", &make_ccp); // Make ConstCharPtr instances Python callable ccp.def("__call__", &ConstCharPtr::operator()); bp::class_("A", "This is class A", bp::init< >()) .def("getName", &get_name) ; }; ############################################################################## # # Boost.Python wrapper: samplemodule.cpp # ############################################################################## #include #include using namespace boost::python; // export function void export_ccp(); // const_char_ptr.cpp BOOST_PYTHON_MODULE(sample) { export_ccp(); }; ############################################################################## # # Python usage # ############################################################################## >>> from sample import * >>> a=A() >>> x=ConstCharPtr() >>> n=a.getName(x) >>> n 1 >>> x() 'A.B' // Bingo! Von: Stefan Seefeld An: cplusplus-sig at python.org Datum: 12.05.2011 15:39 Betreff: Re: [C++-sig] from_python conversion for a const char*& argument Gesendet von: cplusplus-sig-bounces+rainer.kluger=lbbw.de at python.org Hi Rainer, On 2011-05-12 08:57, Rainer Kluger wrote: > > Hi > > I try to expose the following class to python (using gcc 4.5.1, boost > 1_44_0, Python 2.7.1): > > > // C++ > class A > { > public: > int getName(const char*& name); // suppose the implementation > assigns the const char* x = "foo"; variable to the ref-pointer > } > > I'd like to expose this API as is and not alter the method interface so I > think I'll have to do s.th. like this: > > ==================================== > > # intended usage in Python: > >>>> import test >>>> a = A() >>>> x = const_char_ptr_ref() >>>> y = a.getName(x) >>>> x() > 'foo' > > ==================================== > > > I followed the "recipes" of the python test example, realizing that > - m1.cpp > -extract.cpp > and tried to get the things together, knowing that those example do not > really match my use case. > > My understanding for what I need to do, are the following steps: > > (1) encapsulate the const char*& variable in a container class, let's > say ConstCharPtrRef"" > (2) expose the ConstCharPtrRef class to Python > (3) provide a factory function for the class exposed in the step (2) > (4) expose the factory function from step (3) Right. > (5) provide an extractor function which extracts the "const char*&" > variable from within the given python argument > (6) register a from_python converter for the given type and the > provided extractor function I'm not sure I understand the need for (5) and (6). If you have defined a ConstCharPtrRef C++ type, and exported it to Python, you will already have converters between the two, so from within C++ you then can simply access the ConstCharPtrRef members by whatever API you define in C++. There is no need for extra converters at that point. I'm also not sure the reference needs to be captured at that level, as that's more of a call policy (for the A::getName member function) than an aspect of the type. Here is how I would approach it (caution: fully untested code): struct ConstCharPtr { char const *x; }; // A factory... ConstCharPtr make_ccp(...); // the A::getName, adjusted to work with a ConstCharPtr argument int get_name(A const &a, ConstCharPtr const &ccp) { return a.getName(ccp);} BOOST_PYTHON_MODULE(test) { // Export the ConstCharPtr type... class_ ccp("ConstCharPtr", no_init); // ...and define how to construct it ccp.def("__init__", make_cpp); // Export the A type... class_ a("A"); // and export the getName member function a.def("getName", get_name); //... } I believe this should actually do what you want. Stefan -- ...ich hab' noch einen Koffer in Berlin... _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org http://mail.python.org/mailman/listinfo/cplusplus-sig ---------------------------------------------------------------------------------- LBBW-Info IT-Security: Die Nachricht war weder verschluesselt noch digital unterschrieben. Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From kir at lapshin.net Tue May 17 15:48:55 2011 From: kir at lapshin.net (Kirill Lapshin) Date: Tue, 17 May 2011 21:48:55 +0800 Subject: [C++-sig] [Py++] shared_ptr registration Message-ID: Hello Roman/All, I used Py++ before, and was very happy with it, now I'm working on a new project and trying to introduce Py++ here as well. Naturally I decided to get a latest version (was using a bit outdated version before), and noticed that there was no release in quite some time, so here is a first question: - I've seen a message on this list from Mar 25 saying that project is alive and kicking -- that was quite a relief. Any plans for new release? I've ran into some problems with 1.0, as in Py++ failed with traceback during code generation, so decided to give latest trunk a go (using r1856 atm). This one worked quite a bit better, but I'm running into some weirdness around shared_ptr registration. The codebase I'm wrapping uses them heavily, and for whatever reason some of them get registered, while others not, and any attempt to for example call a function returning such unregistered shared_ptr unsurprisingly results in "No to_python (by-value) converter" error. I've reproduced this problem on small example. Before we go there though I'd like to formulate another question: - How shared_ptr registration supposed to work? Is client expected to manually add calls to bp::register_ptr_to_python, or is it something Py++ supposed to do automagically? Is there any way to force Py++ to add such registration? Here is a promised example: ----------------- #include class foo1; typedef boost::shared_ptr foo1_ptr_t; class foo1 { public: foo1_ptr_t parent() const{return foo1_ptr_t(new foo1);} }; class foo2; typedef boost::shared_ptr foo2_ptr_t; class foo2 { public: virtual ~foo2() {} foo2_ptr_t parent() const{return foo2_ptr_t(new foo2);} }; ------------------ As you can see there are two slightly different cases. The only difference is that foo2_ptr_t is a pointer to const foo2. However from Python I can only call foo1::parent, foo2::parent fails with "No to_python (by-value) converter". The code generated by Py++ is: ------------------ { //::foo1 typedef bp::class_< foo1 > foo1_exposer_t; foo1_exposer_t foo1_exposer = foo1_exposer_t( "foo1" ); bp::scope foo1_scope( foo1_exposer ); { //::foo1::parent typedef ::foo1_ptr_t ( ::foo1::*parent_function_type )( ) const; foo1_exposer.def( "parent" , parent_function_type( &::foo1::parent ) ); } bp::register_ptr_to_python< boost::shared_ptr< foo1 > >(); } bp::class_< foo2 >( "foo2" ) .def( "parent" , (::foo2_ptr_t ( ::foo2::* )( ) const)( &::foo2::parent ) ); ------------------ Why code is so different for these two cases? Why foo2 doesn't have bp::register_to_python? During code generation I get W1040 warnings saying that "shared_ptr The declaration is unexposed, but there are other declartions which refer to it". I get them for both foo1 and foo2, even though Py++ did expose shared_ptr. Why the warning is generated for foo1? While we are on warnings topic, another thing I've noticed. Suppose I have function referring std::vector >, Py++ will happily export this vector of strings but will give it ugly name. If I want to rename it, I can explicitly find declaration .include() and .rename() it. In this case however I start getting some weird warnings: WARNING: std::basic_string,std::allocator > * std::vector >::data() [member function] > compilation error W1050: The function returns "std::basic_string,std::allocator > *" type. You have to specify a > call policies.Be sure to take a look on `Py++` defined call policies WARNING: std::basic_string,std::allocator > const * std::vector >::data() const [member function] > compilation error W1050: The function returns "std::basic_string,std::allocator > const *" type. You have to specify > a call policies.Be sure to take a look on `Py++` defined call policies Regards, Kirill From Holger.Joukl at LBBW.de Tue May 17 16:04:27 2011 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Tue, 17 May 2011 16:04:27 +0200 Subject: [C++-sig] py++ site down? Project moved? Message-ID: Hi, has py++ moved somewhere else? This seems to be unreachable: http://language-binding.net/pyplusplus/pyplusplus.html Regards Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From roman.yakovenko at gmail.com Tue May 17 20:44:32 2011 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Tue, 17 May 2011 21:44:32 +0300 Subject: [C++-sig] py++ site down? Project moved? In-Reply-To: References: Message-ID: On Tue, May 17, 2011 at 5:04 PM, Holger Joukl wrote: > > Hi, > > has py++ moved somewhere else? > > This seems to be unreachable: > > http://language-binding.net/pyplusplus/pyplusplus.html I just redirected "language-binding.net" to http://sourceforge.net/projects/pygccxml/ . It will take some time until the change will be distributed among DNS's. The Py++ site consisted from the documentation and adsense. I remove the latest one and put the updated documentation at: http://sourceforge.net/projects/pygccxml/files/docs-pygccxml-pyplusplus/docs-Mar-17-2011.zip/download HTH -------------- next part -------------- An HTML attachment was scrubbed... URL: From roman.yakovenko at gmail.com Tue May 17 22:27:12 2011 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Tue, 17 May 2011 23:27:12 +0300 Subject: [C++-sig] [Py++] shared_ptr registration In-Reply-To: References: Message-ID: On Tue, May 17, 2011 at 4:48 PM, Kirill Lapshin wrote: > Hello Roman/All, > Hello. > I used Py++ before, and was very happy with it, now I'm working on a new > project and trying to introduce Py++ here as well. Naturally I decided to > get a latest version (was using a bit outdated version before), and noticed > that there was no release in quite some time, so here is a first question: > > - I've seen a message on this list from Mar 25 saying that project is alive > and kicking -- that was quite a relief. Any plans for new release? > Not in a few next months. The SVN contains tested and documented version. It is perfectly save to use it. > > I've ran into some problems with 1.0, as in Py++ failed with traceback > during code generation, so decided to give latest trunk a go (using r1856 > atm). This one worked quite a bit better, but I'm running into some > weirdness around shared_ptr registration. The codebase I'm wrapping uses > them heavily, and for whatever reason some of them get registered, while > others not, and any attempt to for example call a function returning such > unregistered shared_ptr unsurprisingly results in "No to_python (by-value) > converter" error. I've reproduced this problem on small example. Before we > go there though I'd like to formulate another question: > > - How shared_ptr registration supposed to work? Basically Py++ inspects every exported declaration and looks for smart pointers and stl containers. Take a look on creators_factory.types_database_t class and follow it within bpcreator.py Once, it has all the information about the exported types and their dependencies, it starts to "expose". So, you really should not / not supposed to configure something. > Is client expected to manually add calls to bp::register_ptr_to_python, or > is it something Py++ supposed to do automagically? Automagically :-) * http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/smart_pointers_to_be_exported.hpp?revision=1837&view=markup * http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/data/smart_pointers_to_be_exported.cpp?revision=1837&view=markup * http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/unittests/smart_pointers_tester.py?revision=1837&view=markup > Is there any way to force Py++ to add such registration? > Yes: mb = module_builder_t( ... ) foo2 = mb.class_('foo2').add_registration_code( 'bp::register_ptr_to_python< boost::shared_ptr< const foo2 > >();', False ) #add_registration_code is documented However it will not help you :-(, boost 1.42 gives the following error: In file included from /usr/include/boost/python/to_python_indirect.hpp:10:0, from /usr/include/boost/python/converter/arg_to_python.hpp:10, from /usr/include/boost/python/call.hpp:15, from /usr/include/boost/python/object_core.hpp:12, from /usr/include/boost/python/args.hpp:25, from /usr/include/boost/python.hpp:11, from smart_pointers.cpp:8: /usr/include/boost/python/object/pointer_holder.hpp: In member function 'void* boost::python::objects::pointer_holder::holds(boost::python::type_info, bool) [with Pointer = boost::shared_ptr, Value = const smart_pointers::maybebug::foo2]': smart_pointers.cpp:500:1: instantiated from here /usr/include/boost/python/object/pointer_holder.hpp:145:66: error: invalid conversion from 'const void*' to 'void*' /usr/include/boost/python/object/pointer_holder.hpp:145:66: error: initializing argument 1 of 'void* boost::python::objects::find_dynamic_type(void*, boost::python::type_info, boost::python::type_info)' /usr/include/boost/python/object/pointer_holder.hpp:145:66: error: invalid conversion from 'const void*' to 'void*' scons: *** [smart_pointers.os] Error 1 scons: building terminated because of errors. ERROR Many years ago, I submitted a patch to this mailing list which adds support for boost::shared_ptr< const T > to Boost.Python. It looks like it was not applied. Here is a promised example: > ... > > Why code is so different for these two cases? > As you can guess already, this is because of "shared_ptr< const ... >" > Why foo2 doesn't have bp::register_to_python? > Mainly because Boost.Python doesn't support it and/or Py++ doesn't recognize it. > During code generation I get W1040 warnings saying that "shared_ptr > The declaration is unexposed, but there are other declartions which refer to > it". I get them for both foo1 and foo2, even though Py++ did expose > shared_ptr. Why the warning is generated for foo1? > A small mistake in design and/or implementation. Please open a bug on sourceforge with example and I will try to fix it. > While we are on warnings topic, another thing I've noticed. Suppose I have > function referring std::vector >, Py++ will happily export this > vector of strings but will give it ugly name. If I want to rename it, I can > explicitly find declaration .include() and .rename() it. In this case > however I start getting some weird warnings: > Why do you ".include()" it? I could be wrong, but you can just rename it and if you use it in your "interface", Py++ should happily expose it under that name. If not, check Py++ "hints" documentation. You have few ways to provide an alias to the class. HTH -------------- next part -------------- An HTML attachment was scrubbed... URL: From kir at lapshin.net Wed May 18 16:06:38 2011 From: kir at lapshin.net (Kirill Lapshin) Date: Wed, 18 May 2011 22:06:38 +0800 Subject: [C++-sig] [Py++] shared_ptr registration In-Reply-To: References: Message-ID: On 18/05/2011 04:27, Roman Yakovenko wrote: > Not in a few next months. The SVN contains tested and documented > version. It is perfectly save to use it. Fair enough. Just asking. > Basically Py++ inspects every exported declaration and looks for smart > pointers and stl containers. Take a look on > creators_factory.types_database_t class and follow it within bpcreator.py > Once, it has all the information about the exported types and their > dependencies, it starts to "expose". So, you really should not / not > supposed to configure something. Perfect. > Is there any way to force Py++ to add such registration? > > > Yes: > > mb = module_builder_t( ... ) > foo2 = mb.class_('foo2').add_registration_code( > 'bp::register_ptr_to_python< boost::shared_ptr< const foo2 > >();', False ) > #add_registration_code is documented > > However it will not help you :-(, boost 1.42 gives the following error: [snip] > Many years ago, I submitted a patch to this mailing list which adds > support for boost::shared_ptr< const T > to Boost.Python. It looks like > it was not applied. Good news that with added registration for shared_ptr it compiles fine with Boost 1.46.1 and works as expected. So I suppose your patch did get applied. > Why foo2 doesn't have bp::register_to_python? > Mainly because Boost.Python doesn't support it and/or Py++ doesn't > recognize it. So now that Boost.Python supports it, any chance to get Py++ to automagically recognize it too? > During code generation I get W1040 warnings saying that > "shared_ptr The declaration is unexposed, but there are other > declartions which refer to it". I get them for both foo1 and foo2, > even though Py++ did expose shared_ptr. Why the warning is > generated for foo1? > > > A small mistake in design and/or implementation. Please open a bug on > sourceforge with example and I will try to fix it. Ok, that is done now. I also went ahead and opened a bug (or rather a wishlist) for shared_ptr support as well. Hope you don't mind. > > While we are on warnings topic, another thing I've noticed. Suppose > I have function referring std::vector >, Py++ will > happily export this vector of strings but will give it ugly name. If > I want to rename it, I can explicitly find declaration .include() > and .rename() it. In this case however I start getting some weird > warnings: > > > Why do you ".include()" it? I could be wrong, but you can just rename it > and if you use it in your "interface", Py++ should happily expose it > under that name. If not, check Py++ "hints" documentation. You have few > ways to provide an alias to the class. You are right, there is no need to .include it. I've ended up including it just because I have a table driven script which blindly includes and renames whatever listed in the table of classes. I've beefed up this logic a bit and now can avoid unnecessary .include, and as you predicted the warning disappeared. Thank you. In the meantime I've got few more questions for you, if you don't mind. Suppose I have two classes with exactly the same name in different namespaces, e.g. ns1::foo and ns2::foo. I include and rename them to say foo1 and foo2. However wrappers generated by Py++ are called foo_wrapper. Py++ even generates helpful warning saying that wrappers have the same name and it might cause problems. Indeed it does. This is not a huge problem, as I can work around it by modifying wrapper_alias property on both declarations. It would be nice though to handle it automatically -- for instance changing wrapper_alias upon rename, or just generating unique wrapper aliases based on fully qualified original name. Another problem I have, although didn't have a chance to try and reproduce it in simple standalone example, is one declaration generates following warning: WARNING: std::string const & base::name() const [member function] > warning W1049: This method could not be overriden in Python - method returns reference to local variable! After some searching through mail list archives I found your message from awhile back implying, if I understood it correctly, that this warning should only happen what non const reference is returned. Any comments on whether I tripped on a bug or whether it is expected warning? If latter is the case how to make it go away (short of adding to list of warnings to ignore). I can try to prep a simple repro if you wish. The last question I have (at least for now :) ) is: Suppose I have a class with a pure virtual method which I don't want to expose for whatever reason, so I exclude this method, right? Now generated code fails to compile because it fails to instantiate an abstract wrapper. Any ideas how to solve/work around this? Thank you so much for your support. Cheers, Kirill From roman.yakovenko at gmail.com Wed May 18 22:57:52 2011 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Wed, 18 May 2011 23:57:52 +0300 Subject: [C++-sig] [Py++] shared_ptr registration In-Reply-To: References: Message-ID: On Wed, May 18, 2011 at 5:06 PM, Kirill Lapshin wrote: > On 18/05/2011 04:27, Roman Yakovenko wrote: >> >> Many years ago, I submitted a patch to this mailing list which adds >> support for boost::shared_ptr< const T > to Boost.Python. It looks like >> it was not applied. > > Good news that with added registration for shared_ptr it compiles fine with Boost 1.46.1 and works as expected. So I suppose your patch did get applied. Cool. It looks like I have to fix Py++. >> ? ?Why foo2 doesn't have bp::register_to_python? >> Mainly because Boost.Python doesn't support it and/or Py++ doesn't >> recognize it. > > So now that Boost.Python supports it, any chance to get Py++ to automagically recognize it too? Yes of course. I'll fix this bug, but it will not happen this or next week so. >> ? ?During code generation I get W1040 warnings saying that >> ? ?"shared_ptr The declaration is unexposed, but there are other >> ? ?declartions which refer to it". I get them for both foo1 and foo2, >> ? ?even though Py++ did expose shared_ptr. Why the warning is >> ? ?generated for foo1? >> >> >> A small mistake in design and/or implementation. Please open a bug on >> sourceforge with example and I will try to fix it. > > Ok, that is done now. I also went ahead and opened a bug (or rather a wishlist) for shared_ptr support as well. Hope you don't mind. The opposite - it is very good. If Py++ will report a lot of irrelevant warning, the user will skip them, and the goal will not be achieved. I will try to fix it too. > In the meantime I've got few more questions for you, if you don't mind. I don't :-) > Suppose I have two classes with exactly the same name in different namespaces, e.g. ns1::foo and ns2::foo. I include and rename them to say foo1 and foo2. However wrappers generated by Py++ are called foo_wrapper. Py++ even generates helpful warning saying that wrappers have the same name and it might cause problems. Indeed it does. This is not a huge problem, as I can work around it by modifying wrapper_alias property on both declarations. It would be nice though to handle it automatically -- for instance changing wrapper_alias upon rename, or just generating unique wrapper aliases based on fully qualified original name. I understand why you are asking for this, but somehow I feel that this functionality doesn't belong to Py++. The functionality could be easily implemented by the user. You can use "partial_decl_string" or "decl_string" properties for this purpose. "partial_decl_string" - returns fully resolved name ( '::ns1::ns2::klass ), without default template arguments (for known classes of course, mainly from std) > Another problem I have, although didn't have a chance to try and reproduce it in simple standalone example, is one declaration generates following warning: > > WARNING: std::string const & base::name() const [member function] > > warning W1049: This method could not be overriden in Python - method returns reference to local variable! > > After some searching through mail list archives I found your message from awhile back implying, if I understood it correctly, that this warning should only happen what non const reference is returned. Any comments on whether I tripped on a bug or whether it is expected warning? If latter is the case how to make it go away (short of adding to list of warnings to ignore). I can try to prep a simple repro if you wish. This warning is reported for virtual functions (please confirm). Consider, virtual or pure virtual function "wrapper" implementation (the one you define in a class, which derives from boost::python::wrapper), for a class implemented in Python. Boost.Python don't have a place to store "std::string". You call the function, get the result, which lives on stack and then you return it. Once you exit the function, your std::string class instance is destroyed. I suggest you to try a simple example and analyze it. > The last question I have (at least for now :) ) is: It's okey > Suppose I have a class with a pure virtual method which I don't want to expose for whatever reason, so I exclude this method, right? Now generated code fails to compile because it fails to instantiate an abstract wrapper. Any ideas how to solve/work around this? I would like to see the example and to know what compiler do you use. Even, in case Py++ doesn't have this functionality, you can change the function alias to "__DoNotCallThisFunctionXYZ" and use "override_precall_code" and "default_precall_code" functionality (see docs and/or tests) to add a code, which will throw exception. Could this help to solve your problem? > Thank you so much for your support. You are welcome. From kir at lapshin.net Thu May 19 15:23:35 2011 From: kir at lapshin.net (Kirill Lapshin) Date: Thu, 19 May 2011 21:23:35 +0800 Subject: [C++-sig] [Py++] shared_ptr registration In-Reply-To: References: Message-ID: On Thursday, May 19, 2011 04:57 AM, Roman Yakovenko wrote: >> So now that Boost.Python supports it, any chance to get Py++ to automagically recognize it too? > Yes of course. I'll fix this bug, but it will not happen this or next week so. That's great, no pressure, I can register them manually for time being. >> Suppose I have two classes with exactly the same name in different namespaces, e.g. ns1::foo and ns2::foo. I include and rename them to say foo1 and foo2. However wrappers generated by Py++ are called foo_wrapper. Py++ even generates helpful warning saying that wrappers have the same name and it might cause problems. Indeed it does. This is not a huge problem, as I can work around it by modifying wrapper_alias property on both declarations. It would be nice though to handle it automatically -- for instance changing wrapper_alias upon rename, or just generating unique wrapper aliases based on fully qualified original name. > > I understand why you are asking for this, but somehow I feel that this > functionality doesn't belong to Py++. The functionality could be > easily implemented by the user. You can use "partial_decl_string" or > "decl_string" properties for this purpose. > "partial_decl_string" - returns fully resolved name ( > '::ns1::ns2::klass ), without default template arguments (for known > classes of course, mainly from std) Fair enough, it is not a big deal and easy to handle in client code. I don't necessarily agree with your point though. To me the whole wrapper business is implementation detail. Unless I'm missing something it doesn't really matter how wrapper class called, as long as name unique. It would be fairly simple to handle in Py++. In a sense Py++ already goes to great length to detect when wrapper names are not unique and generate warning. Suppose for argument sake Py++ would generate sequential wrapper names, e.g. wrapper1, wrapper2, etc. It would simplify Py++, as there would be no need to detect wrapper name clashes and report warnings. It would make generated code less readable, but with a bit more elaborate naming scheme for wrapper that could be fixed too. Anyways, as I said, I don't have problem with handling it on my side, just sharing my thoughts. >> Another problem I have, although didn't have a chance to try and reproduce it in simple standalone example, is one declaration generates following warning: >> >> WARNING: std::string const& base::name() const [member function] >>> warning W1049: This method could not be overriden in Python - method returns reference to local variable! >> >> After some searching through mail list archives I found your message from awhile back implying, if I understood it correctly, that this warning should only happen what non const reference is returned. Any comments on whether I tripped on a bug or whether it is expected warning? If latter is the case how to make it go away (short of adding to list of warnings to ignore). I can try to prep a simple repro if you wish. > > This warning is reported for virtual functions (please confirm). > Consider, virtual or pure virtual function "wrapper" implementation > (the one you define in a class, which derives from > boost::python::wrapper), for a class implemented in Python. > Boost.Python don't have a place to store "std::string". You call the > function, get the result, which lives on stack and then you return it. > Once you exit the function, your std::string class instance is > destroyed. Makes perfect sense, now what could be done to avoid this warning, other than just disabling the warning? For instance is there a way to tell Py++ hey, look, this method doesn't have to be (or shouldn't be) overrideable in Python? >> Suppose I have a class with a pure virtual method which I don't want to expose for whatever reason, so I exclude this method, right? Now generated code fails to compile because it fails to instantiate an abstract wrapper. Any ideas how to solve/work around this? > > I would like to see the example and to know what compiler do you use. > > Even, in case Py++ doesn't have this functionality, you can change the > function alias to "__DoNotCallThisFunctionXYZ" and use > "override_precall_code" and "default_precall_code" functionality (see > docs and/or tests) to add a code, which will throw exception. > Could this help to solve your problem? Didn't have a chance to create an example today. Will try to post one tomorrow. I'm using gcc 4.4 on Red Hat. All the best, Kirill From kir at lapshin.net Thu May 19 15:27:02 2011 From: kir at lapshin.net (Kirill Lapshin) Date: Thu, 19 May 2011 21:27:02 +0800 Subject: [C++-sig] [Py++] Bug in balanced file writer Message-ID: Hi Roman/all, Today I've found what I believe a bug in balanced file writer. Suppose you have 24 class_creators and trying to split them into 10 buckets. What it currently does is: - how many creators per bucket? 24/10 = 2. uhm, ok. - split_sequence of 24 creators by 2, this returns 12 buckets - is actual number of buckets greater than requested? yes 12>10. Ok lets peel of last bucket and merge it into one before last. Now we've got 11 buckets, and we happily write 11 files... but user asked for 10. Here is a patch to fix it, it simply distributes class creators over buckets in round robin. Any chance to get it applied? --- balanced_files.py.orig 2011-05-19 15:58:24.000000000 +0800 +++ balanced_files.py 2011-05-19 16:44:12.000000000 +0800 @@ -13,7 +13,6 @@ from pygccxml import declarations from pyplusplus import decl_wrappers from pyplusplus import code_creators -from pyplusplus.utils import split_sequence #TODO: to add namespace_alias_t classes class balanced_files_t(multiple_files.multiple_files_t): @@ -52,10 +51,11 @@ class_creators = filter( lambda cc: not cc.declaration.already_exposed , class_creators ) - buckets = split_sequence(class_creators, len(class_creators)/self.number_of_buckets ) - if len(buckets) > self.number_of_buckets: - buckets[len(buckets)-2] += buckets[len(buckets)-1] - buckets = buckets[:len(buckets)-1] + buckets = [[]] * self.number_of_buckets + i = 0 + for c in class_creators: + buckets[i].append( c ) + i = (i+1) % self.number_of_buckets for index, bucket in enumerate( buckets ): From nox at 7bitfaster.de Fri May 20 10:11:30 2011 From: nox at 7bitfaster.de (Nox 7Bitfaster) Date: Fri, 20 May 2011 10:11:30 +0200 (MEST) Subject: [C++-sig] boost.python call_method+thread => wrong exceptionhandling? Message-ID: <201105200811.p4K8BUFv009020@post.webmailer.de> An HTML attachment was scrubbed... URL: -------------- next part -------------- Hi, Im currently working with boost.python and stumpled into a strang behavior. If I call a not existing method with call_method from the main thread everything works as excepted (exception is raised and handled in python) but if I call it from a seperate thread call_method does not raise a c++ runtime exception and the hole program crashes. Both cases can be switched by exchanging do_file1 with do_file2 in the testcode. Has someone an idea what I am doing wrong? Minimal testcase: http://codepad.org/7OMMBZGQ Tested with: boost 1.46.1 (statically linked) and python 2.6.5 under windows vista 32 bit Or as a plain code: #include #include "boost/thread.hpp" #include "boost/python.hpp" using boost::noncopyable; using namespace boost::python; struct Interface { PyObject *self; public: Interface(PyObject* s) : self(s) {} static void Thread(Interface* obj) { obj->func(); } void start_thread(void) { boost::thread::thread(Interface::Thread, this); } void func(void) { PyGILState_STATE gstate = PyGILState_Ensure(); call_method(self, "not_existing_function_should_raise_an_exception"); PyGILState_Release(gstate); } }; // specialize has_back_reference for Entity namespace boost { namespace python { template <> struct has_back_reference : mpl::true_ {}; }} BOOST_PYTHON_MODULE(test) { class_("Interface") .def("start_thread", &Interface::start_thread) .def("func", &Interface::func) ; } int main( int argc, char ** argv ) { PyImport_AppendInittab( "test", &inittest ); Py_Initialize(); PyEval_InitThreads(); const char* do_file1 = "import testnobj = test.Interface()nobj.func()"; //raises the correct exception const char* do_file2 = "import testnobj = test.Interface()nobj.start_thread()nwhile 1: pass"; //raises no exception at all PyRun_SimpleString(do_file2); return 0; } From rwgk at yahoo.com Sat May 21 02:10:31 2011 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Fri, 20 May 2011 17:10:31 -0700 (PDT) Subject: [C++-sig] boost.python call_method+thread => wrong exceptionhandling? In-Reply-To: <201105200811.p4K8BUFv009020@post.webmailer.de> References: <201105200811.p4K8BUFv009020@post.webmailer.de> Message-ID: <937435.2855.qm@web111413.mail.gq1.yahoo.com> A very superficial remark: the boost/python.hpp include has to appear first, before any other includes (this goes back to a Python.h requirement, which has to appear before any system include). I only give a small chance of fixing your problem, but it will also only take a few seconds to try out... Ralf >________________________________ >From: Nox 7Bitfaster >To: cplusplus-sig at python.org >Sent: Friday, May 20, 2011 1:11 AM >Subject: [C++-sig] boost.python call_method+thread => wrong exceptionhandling? > > >Message > > >Hi, >Im currently working with boost.python and stumpled into a strang behavior. If?I call a not existing method with call_method from the main thread everything works as excepted (exception is raised and handled in python)?but if I call it from a seperate thread?call_method does not raise a c++ runtime exception and the hole program crashes. Both cases can be switched by exchanging do_file1 with do_file2 in the testcode. Has someone an?idea what I am doing wrong? >Minimal testcase: http://codepad.org/7OMMBZGQ? >Tested with: boost 1.46.1 (statically linked) and python 2.6.5 under windows vista 32 bit >Or as a plain code: >#include >#include"boost/thread.hpp" >#include"boost/python.hpp" >usingboost::noncopyable; >usingnamespaceboost::python; >structInterface{ >PyObject *self; >public:Interface(PyObject* s) : self(s) {} >staticvoidThread(Interface* obj){ >obj->func(); >} >voidstart_thread(void){ >boost::thread::thread(Interface::Thread, this);} >voidfunc(void){ >PyGILState_STATE gstate = PyGILState_Ensure(); >call_method(self, "not_existing_function_should_raise_an_exception");PyGILState_Release(gstate); >} >};// specialize has_back_reference for Entity >namespaceboost { namespacepython{ >template<> structhas_back_reference : mpl::true_ {};}} >BOOST_PYTHON_MODULE(test) >{ >class_("Interface") >.def("start_thread", &Interface::start_thread) >.def("func", &Interface::func); >} >intmain( intargc, char** argv ){ >PyImport_AppendInittab( "test", &inittest );Py_Initialize(); >PyEval_InitThreads(); >constchar* do_file1 = "import testnobj = test.Interface()nobj.func()"; //raises the correct exception >constchar* do_file2 = "import testnobj = test.Interface()nobj.start_thread()nwhile 1: pass"; //raises no exception at allPyRun_SimpleString(do_file2); >return0;} > >_______________________________________________ >Cplusplus-sig mailing list >Cplusplus-sig at python.org >http://mail.python.org/mailman/listinfo/cplusplus-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From nox at 7bitfaster.de Sat May 21 14:01:02 2011 From: nox at 7bitfaster.de (Nox 7Bitfaster) Date: Sat, 21 May 2011 14:01:02 +0200 (MEST) Subject: [C++-sig] boost.python call_method+thread => wrong exceptionhandling? Message-ID: <201105211201.p4LC12vw021230@post.webmailer.de> An HTML attachment was scrubbed... URL: -------------- next part -------------- I removed the windows.h include and placed the boost/python.hpp include at the first place but the behavior did not change. From roman.yakovenko at gmail.com Sat May 21 21:42:47 2011 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Sat, 21 May 2011 22:42:47 +0300 Subject: [C++-sig] [Py++] Bug in balanced file writer In-Reply-To: References: Message-ID: On Thu, May 19, 2011 at 4:27 PM, Kirill Lapshin wrote: > Hi Roman/all, > > Today I've found what I believe a bug in balanced file writer. > > Suppose you have 24 class_creators and trying to split them into 10 > buckets. What it currently does is: > - how many creators per bucket? 24/10 = 2. uhm, ok. > - split_sequence of 24 creators by 2, this returns 12 buckets > - is actual number of buckets greater than requested? yes 12>10. Ok lets > peel of last bucket and merge it into one before last. > > Now we've got 11 buckets, and we happily write 11 files... but user asked > for 10. > > Here is a patch to fix it, it simply distributes class creators over > buckets in round robin. > Thank you for finding bug and providing a fix! > Any chance to get it applied? > > As in many cases, the answer is "yes" and "no". "Yes" - please open a ticket on sourceforge. I have few interesting plans for version 2.0 which will introduce few breaking changes. I'll be glad to insert your patch too "No" - this is a "silent" breaking change. The developer may not pay attention, that the last file was not updated by the code generator and still to compile it. So, in this specific case, I prefer to keep the whole process backward compatible and introduce the patch. Hope, you understand. -------------- next part -------------- An HTML attachment was scrubbed... URL: From roman.yakovenko at gmail.com Sat May 21 21:53:29 2011 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Sat, 21 May 2011 22:53:29 +0300 Subject: [C++-sig] [Py++] shared_ptr registration In-Reply-To: References: Message-ID: On Thu, May 19, 2011 at 4:23 PM, Kirill Lapshin wrote: > Fair enough, it is not a big deal and easy to handle in client code. I > don't necessarily agree with your point though. To me the whole wrapper > business is implementation detail. Unless I'm missing something it doesn't > really matter how wrapper class called, as long as name unique. It would be > fairly simple to handle in Py++. In a sense Py++ already goes to great > length to detect when wrapper names are not unique and generate warning. > Suppose for argument sake Py++ would generate sequential wrapper names, e.g. > wrapper1, wrapper2, etc. It would simplify Py++, as there would be no need > to detect wrapper name clashes and report warnings. It would make generated > code less readable, but with a bit more elaborate naming scheme for wrapper > that could be fixed too. Anyways, as I said, I don't have problem with > handling it on my side, just sharing my thoughts. > Thanks for sharing. The "wrapper class" is a part of the interface, that's why in my opinion, its name is very important and the developer should spend few more minutes to name them properly. I think, we can add a new flag to module_builder_t, so it would generate unique name for every class. If you like the idea, please open a ticket :-) > > Another problem I have, although didn't have a chance to try and reproduce >>> it in simple standalone example, is one declaration generates following >>> warning: >>> >>> WARNING: std::string const& base::name() const [member function] >>> >>>> warning W1049: This method could not be overriden in Python - method >>>> returns reference to local variable! >>>> >>> >>> After some searching through mail list archives I found your message from >>> awhile back implying, if I understood it correctly, that this warning should >>> only happen what non const reference is returned. Any comments on whether I >>> tripped on a bug or whether it is expected warning? If latter is the case >>> how to make it go away (short of adding to list of warnings to ignore). I >>> can try to prep a simple repro if you wish. >>> >> >> This warning is reported for virtual functions (please confirm). >> Consider, virtual or pure virtual function "wrapper" implementation >> (the one you define in a class, which derives from >> boost::python::wrapper), for a class implemented in Python. >> Boost.Python don't have a place to store "std::string". You call the >> function, get the result, which lives on stack and then you return it. >> Once you exit the function, your std::string class instance is >> destroyed. >> > > Makes perfect sense, now what could be done to avoid this warning, other > than just disabling the warning? For instance is there a way to tell Py++ > hey, look, this method doesn't have to be (or shouldn't be) overrideable in > Python? Unfortunately you will have to disable the warning :-( HTH -------------- next part -------------- An HTML attachment was scrubbed... URL: From kir at lapshin.net Sun May 22 09:32:01 2011 From: kir at lapshin.net (Kirill Lapshin) Date: Sun, 22 May 2011 15:32:01 +0800 Subject: [C++-sig] [Py++] Bug in balanced file writer In-Reply-To: References: Message-ID: On Sunday, May 22, 2011 03:42 AM, Roman Yakovenko wrote: > Suppose you have 24 class_creators and trying to split them into 10 > buckets. What it currently does is: > - how many creators per bucket? 24/10 = 2. uhm, ok. > - split_sequence of 24 creators by 2, this returns 12 buckets > - is actual number of buckets greater than requested? yes 12>10. Ok > lets peel of last bucket and merge it into one before last. > > Now we've got 11 buckets, and we happily write 11 files... but user > asked for 10. > > Here is a patch to fix it, it simply distributes class creators over > buckets in round robin. > > > Thank you for finding bug and providing a fix! > > Any chance to get it applied? > > > As in many cases, the answer is "yes" and "no". > > "Yes" - please open a ticket on sourceforge. I have few interesting > plans for version 2.0 which will introduce few breaking changes. I'll be > glad to insert your patch too Done. > "No" - this is a "silent" breaking change. The developer may not pay > attention, that the last file was not updated by the code generator and > still to compile it. So, in this specific case, I prefer to keep the > whole process backward compatible and introduce the patch. > > Hope, you understand. I understand importance of backward compatibility and your desire to not break compatibility is commendable. That being said I think it is still worth considering fixing it. From my point of view the main point of balanced file writer is to guarantee stable set of output files, so that when one decides to expose one more class he wouldn't have to modify his build system scripts/project files. This bug makes this use case nearly impossible, as for instance in example above, if I expose one more class I'll get 12 output files, as I keep adding classes it will grow to 14 output files when number of exposed classes reaches 28, and then will drop back to 10 when number of classes becomes 30. This is not exactly 'silent' breaking change either, as by default Py++ would delete extra output files and therefore build would fail. If you don't buy these arguments, that's fine, it is your call. Would you be interested in a backward compatible change? For instance have an extra input to module_builder.write_balanced_files which would trigger new splitting logic? If you don't want to clutter interface with extra flags than could you suggest if there is a way to use my own file_writer? I don't see easy way as module_builder.write_balanced_files calls some private methods like __merge_user_code. Cheers, Kirill From kuehn.karl at gmail.com Wed May 25 17:21:32 2011 From: kuehn.karl at gmail.com (Karl Kuehn) Date: Wed, 25 May 2011 08:21:32 -0700 Subject: [C++-sig] compiling python extensions to work for multiple versions of python Message-ID: <61F17E6F-FD7E-4ED4-9681-33DA59D0609F@gmail.com> I am working on compiling a python extension (specifically libtorrent-rasterbar), and would like to wind up with a single executable that I can use on multiple versions of MacOS X (specifically 10.5 and 10.6) without having to force the version of python used on each version (10.5 has 2.5, 10.6 has 2.5 and 2.6, but defaults to 2.6). When I compile using bjam: bjam debug-symbols=on dht-support=on boost=source architecture=combined toolset=darwin address-model=32_64 (with BOOST_BUILD and BOOST_ROOT set approrpiatly) I get a working binary (libtorrent.so) that works great on the platform I am working on, and is a "Universal binary" (ppc/ppc64/i386/x86_64), but when I try to "import" it on the other platform without specifying the version of python I get: Fatal Python error: Interpreter not initialized (version mismatch?) Abort trap Forcing the version of python (so forcing 10.6 to use 2.5) allows the module to be imported and used without problem. After an evening of searching around on mailing lists I almost came to the conclusion that it is not possible to compile python modules that work with multiple versions of python... except that I have built a version of the MySQL module that does exactly that. It runs without complaint on 10.5 (ppc and intel), and 10.6 (intel only), on both python 2.5 and 2.6. So my questions: 1) Is this a limitation of Boost.Python? A trick from MySQL that I can emulate? Am I missing something obvious? I have tried a large number of variations in build commands using combinations of the following: link=shared --layout=versioned --enable-framework --build-type=complete link=shared,static macosx-version=10.5 macosx-version-min=10.5 2) If this is not a possibility with Boost.Python-built modules, what would be the best work-around. I can probably build separate modules with the target python version in their names, and play games with importing them. But that is obviously a less-desired version of things, since it would require: a) multiple 60Meg binaries, and b) having 2 at the moment, and presumably a third when 10.7 ships. Note: I am using Boost 1.46.1 and libtorrent-rasterbar 0.15.6, the latest releases of both as of a couple of days ago. -- Karl Kuehn larkost at softhome.net From wichert at wiggy.net Wed May 25 17:25:16 2011 From: wichert at wiggy.net (Wichert Akkerman) Date: Wed, 25 May 2011 17:25:16 +0200 Subject: [C++-sig] compiling python extensions to work for multiple versions of python In-Reply-To: <61F17E6F-FD7E-4ED4-9681-33DA59D0609F@gmail.com> References: <61F17E6F-FD7E-4ED4-9681-33DA59D0609F@gmail.com> Message-ID: <4DDD1F5C.2010107@wiggy.net> On 5/25/11 17:21 , Karl Kuehn wrote: > 1) Is this a limitation of Boost.Python? A trick from MySQL that I can emulate? Am I missing something obvious? I have tried a large number of variations in build commands using combinations of the following: This is a limitation of python, and I would be very afraid of using a MySQL module if it was compiled for another python version, even if it does load. Python versions do not offer a stable ABI: it will change depending on python version, OS, python configure options, compile flags, etc. You should always try to compile Python modules against the specific Python version you want to use. Wichert. From kuehn.karl at gmail.com Wed May 25 18:45:15 2011 From: kuehn.karl at gmail.com (Karl Kuehn) Date: Wed, 25 May 2011 09:45:15 -0700 Subject: [C++-sig] compiling python extensions to work for multiple versions of python In-Reply-To: <4DDD1F5C.2010107@wiggy.net> References: <61F17E6F-FD7E-4ED4-9681-33DA59D0609F@gmail.com> <4DDD1F5C.2010107@wiggy.net> Message-ID: <1890731E-3A21-4F8C-8E2C-0E6B3304B4E5@gmail.com> On May 25, 2011, at 8:25 AM, Wichert Akkerman wrote: >> 1) Is this a limitation of Boost.Python? A trick from MySQL that I can emulate? Am I missing something obvious? I have tried a large number of variations in build commands using combinations of the following: > > This is a limitation of python, and I would be very afraid of using a MySQL module if it was compiled for another python version, even if it does load. Python versions do not offer a stable ABI: it will change depending on python version, OS, python configure options, compile flags, etc. You should always try to compile Python modules against the specific Python version you want to use. Thanks for the pointer. Obviously not what I wanted to hear, but valuable none-the-less. I will just have to go about compiling the modules with separate names, and just swallow the extra bandwidth required to have multiple binaries. From what you say, it is odd that the MySQL is working as well as it is then. A single compiled binary has been working very reliably for more than 2 years running on 10.4 (since dropped), 10.5, and 10.6? all of which default to different versions of python. Since this is a testing system we would have glaring holes in our data if it was not working reliably. Ah, well? off to start up a few compiles? Thanks much! -- Karl Kuehn larkost at softhome.net -------------- next part -------------- An HTML attachment was scrubbed... URL: From super24bitsound at hotmail.com Wed May 25 23:26:42 2011 From: super24bitsound at hotmail.com (Jay Riley) Date: Wed, 25 May 2011 17:26:42 -0400 Subject: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions Message-ID: Hi all, Hopefully this is the correct way to post a question, If not I apologize as I haven't used mailing lists before. Anyways, here is my problem: I have some class functions that takes in parameters by pointer. However, when I expose and override this class/these functions in boost::python, any changes I make to the parameters are not reflected back to the original, meaning it isn't being passed by pointer as I expect. I previously had them as pass-by-reference, but the same problem still occurred. For an example of what I mean, here's some of my code with unimportant functions and details omitted. class StatusEffect { public: virtual void TickCharacter(Character::BaseCharacter* character, Battles::BattleField *field, int ticks = 1); } //In my Scripting Code header file struct StatusEffectWrapper : Game::StatusEffects::StatusEffect { virtual void TickCharacter(Game::Character::BaseCharacter* character, Game::Battles::BattleField *field, int ticks = 1); virtual void TickCharacterDefault(Game::Character::BaseCharacter* character, Game::Battles::BattleField *field, int ticks = 1); } //In the actual scripting code void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter* character, Game::Battles::BattleField *field, int ticks) { call_method(self, "TickCharacter", character, field, ticks); } void StatusEffectWrapper::TickCharacterDefault(Game::Character::BaseCharacter* character, Game::Battles::BattleField *field, int ticks) { this->StatusEffect::TickCharacter(character, field, ticks); } class_ >("StatusEffect") .def("TickCharacter", &StatusEffect::TickCharacter, &StatusEffectWrapper::TickCharacterDefault) and finally in my Python file, I try and do something like this class StatCreepEffect(StatusEffect): def __init__(self, name, stat_, rate_, descript, uid, turns, prior, canrefresh, poseffect, cats, conds, flgs, sep, pers): StatusEffect.__init__(self, name, descript, uid, turns, prior, canrefresh, poseffect, cats, conds, flgs, sep, pers) self.rate = rate_ self.stat = stat_ def TickCharacter(self, Character, Field, Ticks): if (self.stat == Stat.HP): Character.SetHP(int(round(Character.GetHP() + Character.GetInBattleStat(self.stat) * self.rate))) print("HP Regenerated on character " + Character.GetName() + " New HP: " + str(Character.GetHP())) elif (self.stat == Stat.MP): Character.SetMP(int(round(Character.GetMP() + Character.GetInBattleStat(self.stat) * self.rate))) elif (self.stat == Stat.SP): Character.SetSP(int(round(Character.SetSP() + Character.GetInBattleStat(self.stat) * self.rate))) Regen = StatCreepEffect("Regen", Stat.HP, 0.05, "Regenerates a small amount of HP every turn", PrimaryEngine.GetUID(), 14, 0, True, True, StatusCategories.SC_Regen, 0, StatusFlags.TickEffect, True, StatusPersistence.SP_Timer_Cure) SELibrary.AddPythonEffect(Regen); If it matters, the AddPythonEffect is defined as follows: void StatusEffectsLibrary::AddPythonEffect(boost::shared_ptr effect) { if (effect.get() != nullptr) { NameToSEMap[effect->GetName()] = effect; IDToSEMap[effect->GetUID()] = effect; } } I call the effect like so in my C++ code StatusEffect* regen = game.GetSELibrary().GetStatusEffect(std::string("Regen")); //This simply does a lookup to the NameToSEMap inside the StatusEffectLibrary and does a .get() on the shared_ptr while(character->GetHP() < 600) { regen->TickCharacter(character, field, 1); std::cout << "Character HP: " << character->GetHP() << std::endl; } The script compiles and the effect is added correctly. The print line in the python script prints the correct value. The character object starts with 400 HP and the print line outputs 430. However, the std::cout line in my C++ code always outputs 400, meaning the extra 30 HP added by the script is not reflected back in the original object. For clarity, here's the console output from the above C++ code fragment: HP Regenerated on character Arian New HP: 430 Character HP: 400 I've tried adding boost::non_copyable to the class wrapper but to no effect. I've replaced my pointers with pass by reference to no effect. I've switched between shared_ptr, auto_ptr, and a straight reference for the HeldType to no effect. I can confirm using shared_ptrs over regular pointers in the function does work, but I'd prefer to avoid using them unless there really isn't another way as it would be tedious and time consuming to do so. Can someone tell me how to get the behaviour I expect? To be clear, I'm not looking for the python script to take ownership of any of the parameters, nor will I be returning anything. I just want the changes made in the function to be reflected back to the passed in object. Thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Thu May 26 00:35:31 2011 From: talljimbo at gmail.com (Jim Bosch) Date: Wed, 25 May 2011 15:35:31 -0700 Subject: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions In-Reply-To: References: Message-ID: <4DDD8433.5000202@gmail.com> On 05/25/2011 02:26 PM, Jay Riley wrote: > I have some class functions that takes in parameters by pointer. > However, when I expose and override this class/these functions in > boost::python, any changes I make to the parameters are not reflected > back to the original, meaning it isn't being passed by pointer as I > expect. I previously had them as pass-by-reference, but the same problem > still occurred. For an example of what I mean, here's some of my code > with unimportant functions and details omitted. > I note two spots that bear further investigation. The most likely source is here: > //In the actual scripting code > void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter* > character, Game::Battles::BattleField *field, int ticks) > { > call_method(self, "TickCharacter", character, field, ticks); > } I think there's a very good chance that call_method is copying your character argument when it converts it to Python. Boost.Python is extra-careful about making sure the C++ objects it puts inside Python objects don't turn into dangling references, and it has no way to control the lifetime of a raw pointer or reference. I think you could probably fix this by using shared_ptr here, because Boost.Python does know how to convert that safely to Python without copying the pointee. If that fails, look here: > .def("TickCharacter", &StatusEffect::TickCharacter, > &StatusEffectWrapper::TickCharacterDefault) I've personally never seen this invocation of .def, with two member function pointer arguments. That doesn't necessarily mean it's incorrect, because overloading virtual methods in Python is feature I almost never use, and it seems like there are a half-dozen ways to do it and I never know which one is considered best (perhaps someone else on this list does?). But you might want to make sure this does what you expect. HTH Jim Bosch From charlessolar at gmail.com Thu May 26 01:27:44 2011 From: charlessolar at gmail.com (charlessolar at gmail.com) Date: Wed, 25 May 2011 23:27:44 +0000 Subject: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functio In-Reply-To: <4DDD8433.5000202@gmail.com> Message-ID: <20cf30223f2933c39e04a4220d08@google.com> You may want to try having Py++ do an example wrapping of a header file or two, thats always what I did while I was wrapping parts of my code that I was not sure about the right conventions in boost python. That being said, I have also never tried passing a pointer to python for python to edit and give back. That kind of design kind of stinks to me, I think a better solution would be to write an interface for a python visitor that you could apply to your C++ game objects, but that is not important here. There are a couple of suggestions I can make, if you switch back to passing by reference, you need to remember to use boost::ref() for the arguments you do not want copied. Eg: call_method(self, "TickCharacter", boost::ref( character ), boost:;ref( field ), ticks) I use a constant ref in some of my functions and I found I always need to tell boost about it other wise the object does not work right. Also, why are you using call_method like this? I thought the right way for calling python functions was if( bp::override func = bp::get_override( "TickCharacter" ) ) func( boost::ref( character ), boost::ref( field ), ticks ); That is what I do anyway and it was in the docs at some point. Maybe the difference in calling convention is causing boost python to not accurately pass your pointers? Charles On , Jim Bosch wrote: > On 05/25/2011 02:26 PM, Jay Riley wrote: > I have some class functions that takes in parameters by pointer. > However, when I expose and override this class/these functions in > boost::python, any changes I make to the parameters are not reflected > back to the original, meaning it isn't being passed by pointer as I > expect. I previously had them as pass-by-reference, but the same problem > still occurred. For an example of what I mean, here's some of my code > with unimportant functions and details omitted. > I note two spots that bear further investigation. > The most likely source is here: > //In the actual scripting code > void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter* > character, Game::Battles::BattleField *field, int ticks) > { > call_method(self, "TickCharacter", character, field, ticks); > } > I think there's a very good chance that call_method is copying your > character argument when it converts it to Python. Boost.Python is > extra-careful about making sure the C++ objects it puts inside Python > objects don't turn into dangling references, and it has no way to control > the lifetime of a raw pointer or reference. I think you could probably > fix this by using shared_ptr here, because Boost.Python does know how to > convert that safely to Python without copying the pointee. > If that fails, look here: > .def("TickCharacter", &StatusEffect::TickCharacter, > &StatusEffectWrapper::TickCharacterDefault) > I've personally never seen this invocation of .def, with two member > function pointer arguments. That doesn't necessarily mean it's incorrect, > because overloading virtual methods in Python is feature I almost never > use, and it seems like there are a half-dozen ways to do it and I never > know which one is considered best (perhaps someone else on this list > does?). But you might want to make sure this does what you expect. > HTH > Jim Bosch > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From super24bitsound at hotmail.com Thu May 26 01:33:27 2011 From: super24bitsound at hotmail.com (Jay Riley) Date: Wed, 25 May 2011 19:33:27 -0400 Subject: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions In-Reply-To: <4DDD8433.5000202@gmail.com> References: , <4DDD8433.5000202@gmail.com> Message-ID: Your reply actually helped alot. I hadn't considered that the call_method might have been being used incorrectly. Turns out it was. After looking up the documentation for call_method, by changing void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter* character, Game::Battles::BattleField *field, int ticks) { call_method(self, "TickCharacter", character, field, ticks); } to this void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter* character, Game::Battles::BattleField *field, int ticks) { call_method(self, "TickCharacter", ptr(character), ptr(field), ticks); } I get the behave I expected. Thanks! > Date: Wed, 25 May 2011 15:35:31 -0700 > From: talljimbo at gmail.com > To: cplusplus-sig at python.org > Subject: Re: [C++-sig] boost::python - how to pass by reference/pointer to python overriden class functions > > On 05/25/2011 02:26 PM, Jay Riley wrote: > > I have some class functions that takes in parameters by pointer. > > However, when I expose and override this class/these functions in > > boost::python, any changes I make to the parameters are not reflected > > back to the original, meaning it isn't being passed by pointer as I > > expect. I previously had them as pass-by-reference, but the same problem > > still occurred. For an example of what I mean, here's some of my code > > with unimportant functions and details omitted. > > > > I note two spots that bear further investigation. > > The most likely source is here: > > > //In the actual scripting code > > void StatusEffectWrapper::TickCharacter(Game::Character::BaseCharacter* > > character, Game::Battles::BattleField *field, int ticks) > > { > > call_method(self, "TickCharacter", character, field, ticks); > > } > > I think there's a very good chance that call_method is copying your > character argument when it converts it to Python. Boost.Python is > extra-careful about making sure the C++ objects it puts inside Python > objects don't turn into dangling references, and it has no way to > control the lifetime of a raw pointer or reference. I think you could > probably fix this by using shared_ptr here, because > Boost.Python does know how to convert that safely to Python without > copying the pointee. > > If that fails, look here: > > > .def("TickCharacter", &StatusEffect::TickCharacter, > > &StatusEffectWrapper::TickCharacterDefault) > > I've personally never seen this invocation of .def, with two member > function pointer arguments. That doesn't necessarily mean it's > incorrect, because overloading virtual methods in Python is feature I > almost never use, and it seems like there are a half-dozen ways to do it > and I never know which one is considered best (perhaps someone else on > this list does?). But you might want to make sure this does what you > expect. > > HTH > > Jim Bosch > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: