From tbrooks.mail at gmail.com Mon Jul 2 00:36:16 2007 From: tbrooks.mail at gmail.com (Tom Brooks) Date: Sun, 1 Jul 2007 18:36:16 -0400 Subject: [C++-sig] Handle class and lifetime In-Reply-To: References: Message-ID: I'm a beginner at this but, I find that it's easiest and most effective to simply use boost::shared_ptr to hold anything. You can get Python to hold it in a boost::shared_ptr, too, by putting boost::shared_ptr into the template of the "graph" class_. Like so: class_ > pyg("graph") Then, just use boost::shared_ptr or boost::weak_ptr everywhere in C++ that you need to reference the graph. It's straight-forward and I find that it eliminates the ownership-struggle between Python and C++ that I frequently get. Hope that helps you. On 6/29/07, Anders Wang Kristensen wrote: > > Hi, > > I'm a little confused by the call policies in boost python. > > I have something like the following situation: > > struct vertex > { > graph* p; > int x; > }; > > struct graph > { > vertex get_vertex(int i); > }; > > and in my python module: > > class_ pyg("graph") > pyg.def("get_vertex", &graph::get_vertex, > with_custodian_and_ward_postcall<0,1>()); > > Is this the right way to make sure that graphs wont be destroyed while > there are still vertices around which reference these graphs? > > Thanks! > > -- Anders > > _______________________________________________ > C++-sig mailing list > C++-sig at python.org > http://mail.python.org/mailman/listinfo/c++-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From roman.yakovenko at gmail.com Mon Jul 2 06:15:26 2007 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Mon, 2 Jul 2007 07:15:26 +0300 Subject: [C++-sig] Handle class and lifetime In-Reply-To: References: Message-ID: <7465b6170707012115w64670f47v5d13fe0fd66496c8@mail.gmail.com> On 6/30/07, Anders Wang Kristensen wrote: > Hi, > > I'm a little confused by the call policies in boost python. > > I have something like the following situation: > > struct vertex > { > graph* p; > int x; > }; > > struct graph > { > vertex get_vertex(int i); > }; > > and in my python module: > > class_ pyg("graph") > pyg.def("get_vertex", &graph::get_vertex, > with_custodian_and_ward_postcall<0,1>()); > > Is this the right way to make sure that graphs wont be destroyed while > there are still vertices around which reference these graphs? I think that you are right. You always can check yourself be testing reference count sys.getrefcount -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From j.reid at mail.cryst.bbk.ac.uk Mon Jul 2 11:05:03 2007 From: j.reid at mail.cryst.bbk.ac.uk (John Reid) Date: Mon, 02 Jul 2007 10:05:03 +0100 Subject: [C++-sig] Raoul Gough's indexing suite Message-ID: I've downloaded Raoul Gough's indexing suite code into my boost/python/suite/indexing directory. I got it from http://svn.boost.org/svn/boost/sandbox/boost/python/suite/indexing/ Now the following code does not compile as advertised on: http://boost-sandbox.sourceforge.net/python/indexing/containers.html #include using namespace boost; using namespace boost::python; using namespace indexing; void export_container() { class_< std::vector > ("vector_int") .def (container_suite< std::vector >()); } The error complains that def does not have the right number of arguments. I'm using VC++ express on XP with Boost 1.34.0 Also I had to add a missing typename to line 75 of container_proxy.hpp: typedef typename container_proxy::value_type value_type; Did I get the wrong version? Where should I get the code from? John. From roman.yakovenko at gmail.com Mon Jul 2 11:17:46 2007 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Mon, 2 Jul 2007 12:17:46 +0300 Subject: [C++-sig] Raoul Gough's indexing suite In-Reply-To: References: Message-ID: <7465b6170707020217u3b98aa63k34e353d4841bbca8@mail.gmail.com> On 7/2/07, John Reid wrote: > I've downloaded Raoul Gough's indexing suite code into my > boost/python/suite/indexing directory. I got it from > http://svn.boost.org/svn/boost/sandbox/boost/python/suite/indexing/ > > Now the following code does not compile as advertised on: > http://boost-sandbox.sourceforge.net/python/indexing/containers.html > > > #include > > using namespace boost; > using namespace boost::python; > using namespace indexing; > > void export_container() > { > class_< std::vector > ("vector_int") > .def (container_suite< std::vector >()); > } > > The error complains that def does not have the right number of arguments. > > > > I'm using VC++ express on XP with Boost 1.34.0 > > Also I had to add a missing typename to line 75 of container_proxy.hpp: > typedef typename container_proxy::value_type value_type; > > > Did I get the wrong version? Where should I get the code from? You can get it from me: http://language-binding.net/pyplusplus/documentation/containers.html SVN: http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ Pay attention: this is a modified version of the original source code. It contains few bug fixes and support for new containers. If you are going to play with it I suggest you to install Py++ and see what code it generates. -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From Mark.English at rbccm.com Mon Jul 2 11:23:58 2007 From: Mark.English at rbccm.com (English, Mark) Date: Mon, 2 Jul 2007 10:23:58 +0100 Subject: [C++-sig] Raoul Gough's indexing suite Message-ID: > -----Original Message----- > From: c++-sig-bounces at python.org [mailto:c++-sig-bounces at python.org]On > Behalf Of John Reid > Sent: 02 July 2007 10:05 > To: c++-sig at python.org > Subject: [C++-sig] Raoul Gough's indexing suite > > > I've downloaded Raoul Gough's indexing suite code into my > boost/python/suite/indexing directory. I got it from > http://svn.boost.org/svn/boost/sandbox/boost/python/suite/indexing/ > > Now the following code does not compile as advertised on: > http://boost-sandbox.sourceforge.net/python/indexing/containers.html > It doesn't look very up to date, but I did notice it in use in Python-Ogre which is actively maintained. For example here: http://python-ogre.org/browser/trunk/python-ogre/boost/libs/python/src/indexing If you don't get any joy here, you could try asking them. E.g. http://groups.google.com/group/python-ogre-developers Not knowing anything about Python-Ogre or the indexing suite, that could all be misinformation though. Cheers, Mark ______________________________________________________________________ This email is intended only for the use of the individual(s) to whom it is addressed and may be privileged and confidential. Unauthorised use or disclosure is prohibited.If you receive This e-mail in error, please advise immediately and delete the original message. This message may have been altered without your or our knowledge and the sender does not accept any liability for any errors or omissions in the message. From j.reid at mail.cryst.bbk.ac.uk Mon Jul 2 12:17:01 2007 From: j.reid at mail.cryst.bbk.ac.uk (John Reid) Date: Mon, 02 Jul 2007 11:17:01 +0100 Subject: [C++-sig] Raoul Gough's indexing suite In-Reply-To: <7465b6170707020217u3b98aa63k34e353d4841bbca8@mail.gmail.com> References: <7465b6170707020217u3b98aa63k34e353d4841bbca8@mail.gmail.com> Message-ID: Roman Yakovenko wrote: > You can get it from me: > http://language-binding.net/pyplusplus/documentation/containers.html > SVN: > http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ > > Pay attention: this is a modified version of the original source code. > It contains few bug fixes and support for new containers. > > If you are going to play with it I suggest you to install Py++ and see > what code it generates. > That works just fine now. I will try and get around to install Py++. Thanks, John. From Lawrence.Spector at CanfieldSci.com Mon Jul 2 15:45:44 2007 From: Lawrence.Spector at CanfieldSci.com (Lawrence Spector) Date: Mon, 2 Jul 2007 09:45:44 -0400 Subject: [C++-sig] Multiple Interpreters Using Boost.Python? In-Reply-To: <7465b6170706281141y447da08buf3a7c2cd270b7723@mail.gmail.com> References: <379940.27910.qm@web31106.mail.mud.yahoo.com> <7465b6170706281141y447da08buf3a7c2cd270b7723@mail.gmail.com> Message-ID: Hello again. I know by default, the Python API only supports one interpreter per process, but I believe there is a way to configure it for multiple interpreters. If so, does Boost.Python support this? Is there any additional steps for Boost.Python to do this, or is it automatic? Let me know. Thanks, Lawrence From seefeld at sympatico.ca Mon Jul 2 16:05:57 2007 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Mon, 02 Jul 2007 10:05:57 -0400 Subject: [C++-sig] Multiple Interpreters Using Boost.Python? In-Reply-To: References: <379940.27910.qm@web31106.mail.mud.yahoo.com> <7465b6170706281141y447da08buf3a7c2cd270b7723@mail.gmail.com> Message-ID: <46890645.9070609@sympatico.ca> Lawrence Spector wrote: > Hello again. I know by default, the Python API only supports one > interpreter per process, but I believe there is a way to configure it > for multiple interpreters. If so, does Boost.Python support this? Is > there any additional steps for Boost.Python to do this, or is it > automatic? What do you mean by 'interpreter' ? What's your use case, exactly ? There is a single 'python runtime' linked to your program, and some global variables are shared throughout the process (such as the GIL (Global Interpreter Lock)). However, if all you want is execute ('interpret') multiple python scripts from within your application without them interfering with each other, that's quite trivial to do with the existing API. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From Lawrence.Spector at CanfieldSci.com Mon Jul 2 16:37:25 2007 From: Lawrence.Spector at CanfieldSci.com (Lawrence Spector) Date: Mon, 2 Jul 2007 10:37:25 -0400 Subject: [C++-sig] Multiple Interpreters Using Boost.Python? In-Reply-To: <46890645.9070609@sympatico.ca> References: <379940.27910.qm@web31106.mail.mud.yahoo.com> <7465b6170706281141y447da08buf3a7c2cd270b7723@mail.gmail.com> <46890645.9070609@sympatico.ca> Message-ID: Yes, what I meant by multiple interpreters is that I can execute multiple scripts from the same program with no interference. How can this be done? Also, you meant a global interpreter lock. Does this mean that two different scripts won't technically execute at the same time? I would like this to scale to multiple processors well. Thanks, Lawrence -----Original Message----- From: c++-sig-bounces at python.org [mailto:c++-sig-bounces at python.org] On Behalf Of Stefan Seefeld Sent: Monday, July 02, 2007 10:06 AM To: Development of Python/C++ integration Subject: Re: [C++-sig] Multiple Interpreters Using Boost.Python? Lawrence Spector wrote: > Hello again. I know by default, the Python API only supports one > interpreter per process, but I believe there is a way to configure it > for multiple interpreters. If so, does Boost.Python support this? Is > there any additional steps for Boost.Python to do this, or is it > automatic? What do you mean by 'interpreter' ? What's your use case, exactly ? There is a single 'python runtime' linked to your program, and some global variables are shared throughout the process (such as the GIL (Global Interpreter Lock)). However, if all you want is execute ('interpret') multiple python scripts from within your application without them interfering with each other, that's quite trivial to do with the existing API. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... _______________________________________________ C++-sig mailing list C++-sig at python.org http://mail.python.org/mailman/listinfo/c++-sig From seefeld at sympatico.ca Mon Jul 2 17:08:33 2007 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Mon, 02 Jul 2007 11:08:33 -0400 Subject: [C++-sig] Multiple Interpreters Using Boost.Python? In-Reply-To: References: <379940.27910.qm@web31106.mail.mud.yahoo.com> <7465b6170706281141y447da08buf3a7c2cd270b7723@mail.gmail.com> <46890645.9070609@sympatico.ca> Message-ID: <468914F1.5000501@sympatico.ca> Lawrence Spector wrote: > Yes, what I meant by multiple interpreters is that I can execute > multiple scripts from the same program with no interference. How can > this be done? Look up the docs for boost::python::exec_file(). You pass dictionaries containing the environment to be seen during execution. These can be distinct objects for the different scripts. > Also, you meant a global interpreter lock. Does this mean that two > different scripts won't technically execute at the same time? I would > like this to scale to multiple processors well. http://docs.python.org/api/threads.html Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From jeff.webb at nta-inc.net Mon Jul 2 19:58:57 2007 From: jeff.webb at nta-inc.net (Jeff Webb) Date: Mon, 02 Jul 2007 12:58:57 -0500 Subject: [C++-sig] using intrusive_ptr objects with boost python In-Reply-To: <7465b6170706231108x152b8174xc90e0fb06904aec6@mail.gmail.com> References: <46783D75.4050600@nta-inc.net> <7465b6170706231108x152b8174xc90e0fb06904aec6@mail.gmail.com> Message-ID: <46893CE1.3020504@nta-inc.net> Roman Yakovenko wrote: > On 6/19/07, Jeff Webb ...After quite a bit of reading, experimentation, and > contemplation, it appears to me that using reference-counting smart > pointers in C++ would be the most consistent and seamless way to provide > a rich python binding for this type of library. Do others agree with > this assertion? > > This is was my experience. Thanks for taking the time to respond, Roman. I appreciate your input. > > Very nice! Unfortunately, I think the overhead of using a shared_ptr > will be too high, since my library will instantiate many small objects. > > It looks like you know what you are doing, but don't you think this is a > premature optimization? Perhaps, but I wanted to make sure that it was *possible* to achieve similar functionality without using shared_ptr, since I think that this may be an issue down the road. I don't want to code myself into a corner. > Just search this mailing list for "intrusive_ptr" and you will find many > useful posts. After some more digging, I found some useful things, but not exactly what I was looking for. I was eventually able to make boost python do what I wanted, but it took some time and a lot of experimentation to wrap my brain around the template magic. I have coded up a library example using intrusive_ptrs that provides similar functionality to the existing shared_ptr support. I think the behavior suits me better than the shared_ptr behavior, because I can choose whether or not the python wrapping of an object is 'persistent'. (Perhaps 'persistent' is not the best word, but I will try to explain what I mean.) The current shared_ptr behavior is what I call 'persistent', meaning that once an object has been wrapped by python, it will never be unwrapped until all shared_ptr references to that object have been deleted. This is good if you have overridden methods in the python object, since 'unwrapping' the object would remove the override behavior. On the other hand, if your goal is to construct and manipulate wrapped C++ objects using python, and then just run some C++ code, then it would be ideal to get rid of all the python wrappers when you are done manipulating the C++ objects from python. In this case, an object sho uld be 'unwrapped' when all python references to the object are deleted. I have implemented this behavior using intrusive_ptrs, and objects with embedded reference count and pyobject pointer. When constructing an object, the user can specify whether the python wrapping should be 'persistent' or not. I have attached the source code for this library, and a simple test program. The output of the test program is shown below. $ ./itest.py info(p): C['p'] info(c): W['c'] p.child: None p.child = c info(p): C['p']-->W['c'] c2 = p.child c2 is c: True deleting c2 deleting c info(p): C['p']-->W['c'] p.child=None [clearing pyobject pointer] ~C('c') info(p): C['p'] ---------------- info(c3): W['c3'] p.child = c3 info(p): C['p']-->W['c3'] deleting c3 [clearing pyobject pointer] info(p): C['p']-->C['c3'] calling new_C info(c4): C['c4'] p.child = c4 ~C('c3') info(p): C['p']-->C['c4'] done [clearing pyobject pointer] ~C('p') [clearing pyobject pointer] ~C('c4') Note that 'c' is a persistent object and 'c3' is not persistent. Note that when all references to 'c3' are removed, the info method is no longer overridden from python: "info(p): C['p']-->C['c3']". This demonstrates that the python wrapper has been removed, but the C++ object still exists. Perhaps the user should be able to optionally choose the non-persistent behavior for shared_ptr as well. I think it would prove useful for some applications, as I mentioned above. What do you think? I am posting this code with the hope that it will help some other folks who come across this thread. Thanks again for all the work on boost python. -Jeff itest.py ------------------------------------------------ #!/usr/bin/env python from ilib import * class D(C): def info(self): s = "D['%s']" % self.name if self.child: s += '-->' + child.info() return s class W(CWrap): def info(self): s = "W['%s']" % self.name if self.child: s += '-->' + child.info() return s def main(): p = D('p') c = W('c', persistent=True) print 'info(p):', info(p) print 'info(c):', info(c) print 'p.child:', p.child print 'p.child = c'; p.child = c print 'info(p):', info(p) print 'c2 = p.child'; c2 = p.child print 'c2 is c:', c2 is c print 'deleting c2'; del c2 print 'deleting c'; del c print 'info(p):', info(p) print 'p.child=None'; p.child=None print 'info(p):', info(p) print '----------------' c3 = W('c3', persistent=False) print 'info(c3):', info(c3) print 'p.child = c3'; p.child = c3 print 'info(p):', info(p) print 'deleting c3'; del c3 print 'info(p):', info(p) print 'calling new_C'; c4 = new_C('c4') print 'info(c4):', info(c4) print 'p.child = c4'; p.child = c4 print 'info(p):', info(p) print 'done' main() ilib.cpp ----------------------------------------- #include "intrusive_ptr_python_helper.cpp" #include #include #include #include using namespace std; using namespace boost; using namespace boost::python; namespace bp = boost::python; //------------------------------------------------------------------------------ class RefCounted; namespace boost { void intrusive_ptr_add_ref(RefCounted * ptr); void intrusive_ptr_release(RefCounted * ptr); void intrusive_ptr_set_pyobject(RefCounted * ptr, PyObject * pyobject); void intrusive_ptr_clear_pyobject(RefCounted * ptr); PyObject * intrusive_ptr_get_pyobject(RefCounted * ptr); } class RefCounted { public: RefCounted(int ref_count = 0, bool persistent = false) : ref_count(ref_count), persistent(persistent), pyobject(0) { } virtual ~RefCounted() { } protected: int get_ref_count() { return ref_count; } private: friend void boost::intrusive_ptr_add_ref(RefCounted * ptr); friend void boost::intrusive_ptr_release(RefCounted * ptr); friend void boost::intrusive_ptr_set_pyobject(RefCounted * ptr, PyObject * pyobject); friend void boost::intrusive_ptr_clear_pyobject(RefCounted * ptr); friend PyObject * boost::intrusive_ptr_get_pyobject(RefCounted * ptr); int ref_count; bool persistent; PyObject * pyobject; }; namespace boost { void intrusive_ptr_add_ref(RefCounted * ptr) { if (ptr->persistent) { if (ptr->pyobject && (ptr->ref_count > 0)) { incref(ptr->pyobject); } } ++(ptr->ref_count); } void intrusive_ptr_release(RefCounted * ptr) { if (ptr->persistent) { if (ptr->pyobject) { decref(ptr->pyobject); } } if (--(ptr->ref_count) == 0) delete ptr; } void intrusive_ptr_set_pyobject(RefCounted * ptr, PyObject * pyobject) { if (ptr->persistent) { for (int i=0; iref_count-1; i++) incref(pyobject); } ptr->pyobject = pyobject; } void intrusive_ptr_clear_pyobject(RefCounted * ptr) { cout << "[clearing pyobject pointer]" << endl; ptr->pyobject = 0; } PyObject * intrusive_ptr_get_pyobject(RefCounted * ptr) { return ptr->pyobject; } } //------------------------------------------------------------------------------ class C; typedef intrusive_ptr CPtr; class C : public RefCounted { public: C(const string &name, bool persistent = false) : RefCounted(0, persistent), name(name) { } virtual ~C() { cout << "~C('" << name << "')" << endl; } void set_child(CPtr child) { this->child = child; } CPtr get_child() { return child; } string get_name() { return name; } virtual string info() { string s = "C['" + name + "']"; if (child) { s += "-->" + child->info(); } return s; } private: string name; CPtr child; }; //------------------------------------------------------------------------------ class CWrap : public C { public: CWrap(const string &name, bool persistent = false) : C(name, persistent) { } CWrap(PyObject * py_obj, const string &name, bool persistent = false) : C(name, persistent) { intrusive_ptr_set_pyobject(this, py_obj); } string info() { if (intrusive_ptr_get_pyobject(this)) { return call_method(intrusive_ptr_get_pyobject(this), "info"); } else { return C::info(); } } string default_info() { string s = "CWrap['" + get_name() + "']"; CPtr child = get_child(); if (child) { s += "-->" + child->info(); } return s; } }; //------------------------------------------------------------------------------ string info(CPtr c) { return c->info(); } //------------------------------------------------------------------------------ CPtr new_C(string name) { return CPtr(new C(name)); } object init_C(object py_obj, string name) { object return_value = call_method