From s_sourceforge at nedprod.com Wed Feb 1 18:45:46 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Wed, 01 Feb 2012 17:45:46 -0000 Subject: [C++-sig] EXTERNAL: Re: Odd dlopen behavior In-Reply-To: <86299D4CFE2C1248AA41ACF782B0D143117DDA5B@HDXDSP33.us.lmco.com> References: <86299D4CFE2C1248AA41ACF782B0D143117DD654@HDXDSP33.us.lmco.com>, <4F26642D.301.666A4AEF@s_sourceforge.nedprod.com>, <86299D4CFE2C1248AA41ACF782B0D143117DDA5B@HDXDSP33.us.lmco.com> Message-ID: <4F297A4A.8000.7278AB9B@s_sourceforge.nedprod.com> On 31 Jan 2012 at 16:44, Davidson, Josh wrote: > Ok, well I did figure out the discrepancy between these extensions and > previous extensions that have been built that required setting > RTLD_GLOBAL. What I'm doing for these extensions is instead of building > in all of the original C++ code AND the Py++ generated code into the > extension, I'm only building in the Py++ generated sources and linking > an existing shared library containing the original C++ definitions. Is > this non-standard or bad practice? The big problem with shared objects exporting lots of symbols was that the Linux runtime shared object linker used to have O(N^3) complexity. As a result, every time you ran a program linking to a ginormous shared object you'd get a pause of several seconds as it bound the symbols. Now, some years ago with KDE and OpenOffice taking forever to load, some eyeballs were turned onto this problem and I know they were going to get it down to O(N^2). There was speak of replacing bits with O(N), but it would introduce ABI compat problems among other things. Another angle was making it use multiple cores. My attention ended up moving elsewhere so I have no idea what has happened since. It could still be O(N^2), it could be O(N) or somewhere in between. > One issue with this is I'm now forced to deliver both the Python > extension shared libraries and the original shared libraries. Not a > huge deal, but it does add a little work on the deployment and > maintenance end. On systems with sane DLL designs like Windows and Mac OS X, you'd generally keep the Python bindings separate from the library being bound as it's cleaner and more self-contained. You can also issue smaller self-container ABI compatible releases as hotfixes etc etc. On the insanity that is ELF, generally you can make inter-SO problems go away by linking everything into a ginormous monolithic SO. However you used to get that O(N^3)/O(N^2) problem I mentioned and maybe you still do. So, sometimes you just have to get your hands dirty and start with hack scripts which post-process the SOs to make their symbol tables sane, or write your own SO loader and binder implementation using dlopen() et al and bypass the system linker altogether :) Sadly the ISO standards work to enforce sanity in shared libraries across all platforms got dropped from C11 and C++11, but I certainly will try to push that forward again for C11 TR1 along with a few other items on my shopping list (I'm the ISO SC22 convenor for Ireland, though Ireland is only an observer). The problem, as always, is a lack of sponsorship or funding by anyone who cares enough to have the problem fixed properly - and it is a difficult problem to get correct. In the end, as much as these problems are annoying and cost time to people like you, the cost of fixing them isn't seen as business relevant by those with the resources. HTH, Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From josh.davidson at lmco.com Thu Feb 2 07:00:32 2012 From: josh.davidson at lmco.com (Davidson, Josh) Date: Thu, 02 Feb 2012 06:00:32 +0000 Subject: [C++-sig] EXTERNAL: Re: Odd dlopen behavior In-Reply-To: <4F297A4A.8000.7278AB9B@s_sourceforge.nedprod.com> References: <86299D4CFE2C1248AA41ACF782B0D143117DD654@HDXDSP33.us.lmco.com> <4F26642D.301.666A4AEF@s_sourceforge.nedprod.com> <86299D4CFE2C1248AA41ACF782B0D143117DDA5B@HDXDSP33.us.lmco.com> <4F297A4A.8000.7278AB9B@s_sourceforge.nedprod.com> Message-ID: <86299D4CFE2C1248AA41ACF782B0D143117DDFEA@HDXDSP33.us.lmco.com> Neil, great information, but I did track this problem down to a quirk with Py++. I've had a great deal of trouble finding a reliable way to actually write modules with module_builder. Originally, I had been using split_module but I've run into several cases where it goes off in the weeds and tries to write files that exceed the maximum file length. Generally, this occurs when wrapping classes that go nuts on specialization since module_builder uses the name of the class in the file bindings file. Here is one quick, extreme example where this occurs when trying to wrap members of boost: File "C:\Users\davidsj2\workspace\SimCommon\src\Python\goes\build\bindings.py", line 325, in _generate files = mb.split_module(self._bindingsDir) File "c:\Python26\lib\site-packages\pyplusplus\module_builder\boost_python_builder.py", line 375, in split_module , encoding=self.encoding) File "c:\Python26\lib\site-packages\pyplusplus\file_writers\__init__.py", line 37, in write_multiple_files mfs.write() File "c:\Python26\lib\site-packages\pyplusplus\file_writers\multiple_files.py", line 406, in write self.split_classes() File "c:\Python26\lib\site-packages\pyplusplus\file_writers\multiple_files.py", line 307, in split_classes map( self.split_class, class_creators ) File "c:\Python26\lib\site-packages\pyplusplus\file_writers\multiple_files.py", line 294, in split_class self.split_class_impl( class_creator ) File "c:\Python26\lib\site-packages\pyplusplus\file_writers\multiple_files.py", line 268, in split_class_impl , self.create_function_code( function_name ) ) ) File "c:\Python26\lib\site-packages\pyplusplus\file_writers\multiple_files.py", line 61, in write_file writer.writer_t.write_file( fpath, content, self.files_sum_repository, self.encoding ) File "c:\Python26\lib\site-packages\pyplusplus\file_writers\writer.py", line 150, in write_file f = codecs.open( fpath, 'w+b', encoding ) File "c:\Python26\lib\codecs.py", line 881, in open file = __builtin__.open(filename, mode, buffering) IOError: [Errno 2] No such file or directory: 'C:\\Users\\davidsj2\\workspace\\SimCommon\\build\\win64\\pybindings\\goes\\boost\\dividable2_less__boost_scope_date_time_scope_date_duration_less__boost_scope_date_time_scope_duration_traits_adapted__greater__comma__int_comma__boost_scope_detail_scope_empty_base_less__boost_scope_date_time_scope_date_duration_less__boost_scope_date_time_scope_duration_traits_adapted__greater___greater___greater_.pypp.hpp' make: *** [all] Error 1 After finding references to this problem as far back as 2006, I decided to switch over to balanced_split_module. This has its own set of problems. The first is that it is highly prone to divide by zero errors. One quick way to reproduce this issue is to wrap one class and specify a split count of 2. Obviously not a wise combo, but it's an easy error case that Py++ should handle. So anyways, the root of *this* problem is how balanced_split_module creates its registration functions. For each extension, it creates one register function for each file it writes in the form: void register_classes_() Obviously, these collide when you create more than one extension using balanced_split_module and enable RTLD_GLOBAL. One quick solution to this problem would be to prepend the extension name to the name of the registration functions, e.g.: _register_classes_ Since the module name is used to name the files, its easily accessible and would solve a lot of problems. Of course, if you have modules with the same name in different packages you would run into this again. Josh -----Original Message----- From: cplusplus-sig-bounces+josh.davidson=lmco.com at python.org [mailto:cplusplus-sig-bounces+josh.davidson=lmco.com at python.org] On Behalf Of Niall Douglas Sent: Wednesday, February 01, 2012 10:46 AM To: Development of Python/C++ integration Subject: Re: [C++-sig] EXTERNAL: Re: Odd dlopen behavior On 31 Jan 2012 at 16:44, Davidson, Josh wrote: > Ok, well I did figure out the discrepancy between these extensions and > previous extensions that have been built that required setting > RTLD_GLOBAL. What I'm doing for these extensions is instead of > building in all of the original C++ code AND the Py++ generated code > into the extension, I'm only building in the Py++ generated sources > and linking an existing shared library containing the original C++ > definitions. Is this non-standard or bad practice? The big problem with shared objects exporting lots of symbols was that the Linux runtime shared object linker used to have O(N^3) complexity. As a result, every time you ran a program linking to a ginormous shared object you'd get a pause of several seconds as it bound the symbols. Now, some years ago with KDE and OpenOffice taking forever to load, some eyeballs were turned onto this problem and I know they were going to get it down to O(N^2). There was speak of replacing bits with O(N), but it would introduce ABI compat problems among other things. Another angle was making it use multiple cores. My attention ended up moving elsewhere so I have no idea what has happened since. It could still be O(N^2), it could be O(N) or somewhere in between. > One issue with this is I'm now forced to deliver both the Python > extension shared libraries and the original shared libraries. Not a > huge deal, but it does add a little work on the deployment and > maintenance end. On systems with sane DLL designs like Windows and Mac OS X, you'd generally keep the Python bindings separate from the library being bound as it's cleaner and more self-contained. You can also issue smaller self-container ABI compatible releases as hotfixes etc etc. On the insanity that is ELF, generally you can make inter-SO problems go away by linking everything into a ginormous monolithic SO. However you used to get that O(N^3)/O(N^2) problem I mentioned and maybe you still do. So, sometimes you just have to get your hands dirty and start with hack scripts which post-process the SOs to make their symbol tables sane, or write your own SO loader and binder implementation using dlopen() et al and bypass the system linker altogether :) Sadly the ISO standards work to enforce sanity in shared libraries across all platforms got dropped from C11 and C++11, but I certainly will try to push that forward again for C11 TR1 along with a few other items on my shopping list (I'm the ISO SC22 convenor for Ireland, though Ireland is only an observer). The problem, as always, is a lack of sponsorship or funding by anyone who cares enough to have the problem fixed properly - and it is a difficult problem to get correct. In the end, as much as these problems are annoying and cost time to people like you, the cost of fixing them isn't seen as business relevant by those with the resources. HTH, Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org http://mail.python.org/mailman/listinfo/cplusplus-sig From s_sourceforge at nedprod.com Thu Feb 2 13:03:46 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Thu, 02 Feb 2012 12:03:46 -0000 Subject: [C++-sig] EXTERNAL: Re: Odd dlopen behavior In-Reply-To: <86299D4CFE2C1248AA41ACF782B0D143117DDFEA@HDXDSP33.us.lmco.com> References: <86299D4CFE2C1248AA41ACF782B0D143117DD654@HDXDSP33.us.lmco.com>, <4F297A4A.8000.7278AB9B@s_sourceforge.nedprod.com>, <86299D4CFE2C1248AA41ACF782B0D143117DDFEA@HDXDSP33.us.lmco.com> Message-ID: <4F2A7BA2.9558.7665EBD8@s_sourceforge.nedprod.com> On 2 Feb 2012 at 6:00, Davidson, Josh wrote: > Neil, great information, but I did track this problem down to a quirk with Py++. > [snip] > After finding references to this problem as far back as 2006, I decided > to switch over to balanced_split_module. This has its own set of > problems. The first is that it is highly prone to divide by zero > errors. One quick way to reproduce this issue is to wrap one class and > specify a split count of 2. Obviously not a wise combo, but it's an > easy error case that Py++ should handle. It's been a long time since I used Py++, or indeed BPL. Neither has seen much work done on them in recent years, so I should imagine both will have suffered from a certain amount of bitrot. > So anyways, the root of *this* problem is how balanced_split_module > creates its registration functions. For each extension, it creates one > register function for each file it writes in the form: void > register_classes_() Obviously, these collide when you create more > than one extension using balanced_split_module and enable RTLD_GLOBAL. In my own code, registration functions are always static and pass their own address into the runtime. The runtime does a two-pass initialisation, so first off it eliminates any duplicate registration functions whose address lies within the same DLL. Then and only then does it start complaining. That solution allows you to throw registration functions anywhere and let the runtime sort out what's what. It also lets you operate per-DLL and DLL-specialised registries, something I suggested to Jim Bosch for the next release of BPL a few months back. Knowing which DLL registered what is also very useful for debugging. > One quick solution to this problem would be to prepend the extension > name to the name of the registration functions, e.g.: > _register_classes_ Since the module name is used to name the > files, its easily accessible and would solve a lot of problems. Of > course, if you have modules with the same name in different packages you > would run into this again. This is a bad solution. Firstly, who is to say that type Foo in extension X is or is not the same as type Foo in extension Y even if both have the same type, same length, same traits and live in the same namespace? Yes I know it's a violation of ODR, but in the real world maybe they are the same, and maybe they aren't - people use DLLs to violate ODR all the time, it's one of their big utilities. You need a way of explicitly specifying if the types are equal or aren't. Type registration functions aren't the place to do it, however in my own code I have a concept of "type conversion" registration functions which are used to declare runtime type conversions which kick in if the static (metaprogrammed) type conversions fail. When you say type Foo is the same between registries X and Y it simply bumps that type out of X and Y and into the common parent registry between X and Y, so searches fail in X and Y and jump into the parent where they get resolved as being identical. Now me and Dave Abrahams come to a disagreement after this - he does not feel that there ought to be separate static and runtime type registries. And I can see his point. But we disagree :). I'm very fond of binding together other people's libraries in order to personally avoid writing as much code as possible, so I think in terms of ways of getting other people's code to cooperate, and for that I like to override when needs be which is blatent ODR breaking. Dave has a different approach, and therefore a different philosophy. (And before Dave points out that that wasn't my philosophy in the past, and indeed he once argued with me strenuously to use Boost rather than write my own metaprogramming library, I admit I was wrong and he was right) However, back to the point. It would appear you're running into well known limitations in BPL. Some years ago, people's needs were generally simpler and BPL worked without issue for them. As years move on, more and more complex use cases are becoming common. Bitrot. It consumes all code eventually. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From brandsmeier at gmx.de Thu Feb 2 15:30:00 2012 From: brandsmeier at gmx.de (Holger Brandsmeier) Date: Thu, 2 Feb 2012 15:30:00 +0100 Subject: [C++-sig] make_constructor and BOOST_PYTHON_FUNCTION_OVERLOADS Message-ID: Dear list, how can I combine make_constructor() and BOOST_PYTHON_FUNCTION_OVERLOADS()? I have a static member function in a class that "acts like a constructors", in the sense that it returns a shared pointer to the class. The function has default arguments, thats why I was thinking of using BOOST_PYTHON_FUNCTION_OVERLOADS. For usual member functions I've been using it like this: BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(Operator_apply, apply, 2, 5) [...] .def("apply", &ClassT::apply, Operator_apply() ) So for the constructor I thought probably I should use it like BOOST_PYTHON_FUNCTION_OVERLOADS(fPfemSpace_create, create, 2, 3) [...] .def("__init__", make_constructor( &ClassT::create ), fPfemSpace_create()) but that fails with the error: /usr/include/boost/python/class.hpp:598:37: error: no matching function for call to 'get_signature' name, overloads, *this, detail::get_signature(sig)); I tried out a few other variations for ".def(" but I didn't find the right one yet. Is there a way to combine make_constructor with BOOST_PYTHON_FUNCTION_OVERLOADS? -Holger From talljimbo at gmail.com Thu Feb 2 16:39:15 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Thu, 02 Feb 2012 10:39:15 -0500 Subject: [C++-sig] make_constructor and BOOST_PYTHON_FUNCTION_OVERLOADS In-Reply-To: References: Message-ID: <4F2AAE23.2090708@gmail.com> On 02/02/2012 09:30 AM, Holger Brandsmeier wrote: > Dear list, > > how can I combine make_constructor() and BOOST_PYTHON_FUNCTION_OVERLOADS()? > > I have a static member function in a class that "acts like a > constructors", in the sense that it returns a shared pointer to the > class. The function has default arguments, thats why I was thinking of > using BOOST_PYTHON_FUNCTION_OVERLOADS. > > For usual member functions I've been using it like this: > BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(Operator_apply, apply, 2, 5) > [...] > .def("apply",&ClassT::apply, Operator_apply() ) > > So for the constructor I thought probably I should use it like > BOOST_PYTHON_FUNCTION_OVERLOADS(fPfemSpace_create, create, 2, 3) > [...] > .def("__init__", make_constructor(&ClassT::create ), > fPfemSpace_create()) > > but that fails with the error: > /usr/include/boost/python/class.hpp:598:37: error: no matching > function for call to 'get_signature' > name, overloads, *this, detail::get_signature(sig)); > > I tried out a few other variations for ".def(" but I didn't find the > right one yet. Is there a way to combine make_constructor with > BOOST_PYTHON_FUNCTION_OVERLOADS? > I've always used arg lists rather than the OVERLOADS macros to deal with default arguments, and I can confirm that those work with make_constructor: bp::make_construct(&ClassT::create, bp::default_call_policies(), (bp::arg("a"), bp::arg(b")=0)) The advantage is that your Python users get to use keyword args; the disadvantage is that you have to write them out (along with the default value). I have noticed, however, that make_constructor isn't as flexible as make_function about the order of its own optional arguments - if you want to pass args, you also have to pass default_call_policies as the second argument, as I did above. The same trick might work for the OVERLOADS macros, if you'd prefer to use those. Jim From brandsmeier at gmx.de Fri Feb 3 20:26:59 2012 From: brandsmeier at gmx.de (Holger Brandsmeier) Date: Fri, 3 Feb 2012 20:26:59 +0100 Subject: [C++-sig] storing weak_ptr to this in classes extended from python In-Reply-To: References: Message-ID: Dear list, Note: I am actually using Teuchos::RCP instead of boost::shared_ptr and boost::weak_ptr, but I already mirrored almost all the special treatments in boost::python, and have been successfully using this for a while. Everything that I write should actually apply to both implementations in the same way. I have a class that needs to store an RCP to itself to pass it along to some classes that it creates, which might later need the RCP to keep the original class alive. In C++ I implemented this by switching the constructor to procted, and by having a static create() method like this: ?static RCP create(RCP< MeshT > mesh, Point pol_deg_default) ?{ ? ?RCP ret(new PfemSpaceT(mesh, pol_deg_default)); ? ?ret->setThisRCP(ret); ? ?ret->init(); ? ?return ret; ?} The function setThisRCP() stores the shared point as a weak pointer. When you call getThisRCP() the a strong RCP is returned from this weak RCP. This is what you want as otherwise the class will never be deleted. The problem starts now as the same class is extended from python. I mirror the same implementation in python as: ?def __init__(self, mesh, pol_deg, cb_setDofEntity): ? ?super(PfemSpaceStaticCond, self).__init__(mesh, pol_deg, False) ? ?self.setThisRCP(self) ? ?self.init() Now I get serious problems with this call: ? ?self.setThisRCP(self) My understanding is, that the following happens: ?1) the from-python converter creates an RCP with the use count of one and a custom deletor (boost::python::converter::shared_ptr_deleter). ?2) the function setThisRCP gets called and stores a weak RCP ?3) the control gets back to python and the use count of the RCP drops to zero. The custom deletor dereferences the use count in python. Later calls to getThisRCP() that turn the weak RCP into a strong RCP are then invalid. When python no longer references the class, it gets deleted. Some C++ parts that used the RCP from getThisRCP() work on a class that has been deleted and at some point the program crashes. (Here maybe if I would use boost the program would crash already earlier). Do you have an idea how I can use the thisRCP paradigm together with C++ classes that are extended from python? Thanks for any advice, Holger From brandsmeier at gmx.de Fri Feb 3 21:13:48 2012 From: brandsmeier at gmx.de (Holger Brandsmeier) Date: Fri, 3 Feb 2012 21:13:48 +0100 Subject: [C++-sig] make_constructor and extending from python Message-ID: Dear list, how can I have a static member function that acts like a constructor in a Wrapper, i.e. I have a class that I want to extend from python? I have a wrapper class around a class (PfemSpace) that has this static member functions: static RCP create( ... ) { RCP ret(new PfemSpaceWrapperT( ... )); ret->setThisRCP(ret); return ret; } This method is exported via .def("__init__", make_constructor( &ClassWrapperT::create ) ) This works without errors, but when I use this from python via class PfemSpaceStaticCond(PfemSpace): def __init__(self, ...): super(PfemSpaceStaticCond, self).__init__(...) def visit_createFemData(self): print '## [PfemSpaceStaticCond_init] visit_createFemData' Here I call the constructor that I exported via make_constructor in __init__(). Unfortunately the member function visit_createFemData that I override in python never gets called. When I use a "real" constructor instead of the one exported via make_constructor() then the function visit_createFemData gets called. I assume the error is somewhere that with a real constructor boost::python can call an inplace constructor, while with my make_constructor it can't. Is there some way to have the needed functionality with make_constructor? Note: today I came to the conclusion that I a read constructor instead of a static member function is not an option for me due to the issues I wrote in my previous mail "storing weak_ptr to this in classes extended from python". That previous mail is actually from yesterday, but it originally got rejected because I send it from a wrong email address, so I'm sorry that you are receiving two mails from me today. For a moment I was very happy that this "static constructor-like function in the Wrapper class" would solve all my problems, but I am still missing something. Thanks for your help, Holger From adam.preble at gmail.com Sat Feb 4 17:11:15 2012 From: adam.preble at gmail.com (Adam Preble) Date: Sat, 4 Feb 2012 10:11:15 -0600 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? Message-ID: I am trying to run this on Win32, using Stackless Python 2.6, and Boost 1.47. I am having a problem with passing a shared_ptr of a C++ object implementing a C++ interface to a wrapper of a Python object implementing a C++ interface. That's a mouthful, so I thought I'd simplify the code. The thing is, I can't seem to get the wrapping correct. If I can get this going, I can start adding complication to it and see what the problem really is. Basically, I have an interface that works with itself for communicating stuff. There is a C implementation of it, and a Python implementation of it. I have to write a wrapper. The Python implementation can communicate with instances of the C implementation, but not vice versa. When the C implementation calls the Python one, I get: Traceback (most recent call last): File "", line 1, in TypeError: No to_python (by-value) converter found for C++ type: class boost::shared_ptr The magic call is in the python script. Python script http://pastebin.com/DJ65tKEW Header http://pastebin.com/6BX8e6KA Source (Boost.Python declarations here) http://pastebin.com/s5YC1hyC I have done similar wrappers like this before so I'm just befuddled about why this time it hates me. The shared_ptrs are part of the larger problem I'm trying to reproduce, so I want to incorporate them into this test program. In the larger program, while doing a get_override() call on a wrapper to an interface, I get a segfault inside the Python DLLs while it tries to convert the shared_ptr. The last line I can see i shared_ptr_to_python.hpp: return converter::registered const&>::converters.to_python(&x); That's what I am eventually hoping to reproduce by shooting back and forth all this stuff. -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Sat Feb 4 17:40:54 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 04 Feb 2012 11:40:54 -0500 Subject: [C++-sig] storing weak_ptr to this in classes extended from python In-Reply-To: References: Message-ID: <4F2D5F96.9000601@gmail.com> On 02/03/2012 02:26 PM, Holger Brandsmeier wrote: > Dear list, > > Note: I am actually using Teuchos::RCP instead of boost::shared_ptr > and boost::weak_ptr, but I already mirrored almost all the special > treatments in boost::python, and have been successfully using this for > a while. Everything that I write should actually apply to both > implementations in the same way. > > I have a class that needs to store an RCP to itself to pass it along > to some classes that it creates, which might later need the RCP to > keep the original class alive. In C++ I implemented this by switching > the constructor to procted, and by having a static create() method > like this: > > static RCP create(RCP< MeshT> mesh, Point > pol_deg_default) > { > RCP ret(new PfemSpaceT(mesh, pol_deg_default)); > > ret->setThisRCP(ret); > ret->init(); > > return ret; > } > > The function setThisRCP() stores the shared point as a weak pointer. > When you call getThisRCP() the a strong RCP is returned from this weak > RCP. This is what you want as otherwise the class will never be > deleted. > > The problem starts now as the same class is extended from python. I > mirror the same implementation in python as: > > def __init__(self, mesh, pol_deg, cb_setDofEntity): > super(PfemSpaceStaticCond, self).__init__(mesh, pol_deg, False) > > self.setThisRCP(self) > self.init() > > Now I get serious problems with this call: > self.setThisRCP(self) > > My understanding is, that the following happens: > 1) the from-python converter creates an RCP with the use count of > one and a custom deletor > (boost::python::converter::shared_ptr_deleter). > 2) the function setThisRCP gets called and stores a weak RCP > 3) the control gets back to python and the use count of the RCP > drops to zero. The custom deletor dereferences the use count in > python. > > Later calls to getThisRCP() that turn the weak RCP into a strong RCP > are then invalid. When python no longer references the class, it gets > deleted. Some C++ parts that used the RCP from getThisRCP() work on a > class that has been deleted and at some point the program crashes. > (Here maybe if I would use boost the program would crash already > earlier). > > Do you have an idea how I can use the thisRCP paradigm together with > C++ classes that are extended from python? > It sounds like you need to ensure that instances created in Python always use an RCP holder (even Python subclasses). I'd have thought passing RCP as the holder template argument in the class_ definition would do that, but I'm not that familiar with how smart-pointer holders interact with wrapper classes. The second part is that you'd need to make sure #1 is replaced by having Boost.Python look in the holder, find that it has an internal RCP, and use that instead of creating a new temporary one. I know Boost.Python can do that with shared_ptr at least some of the time, so I assume you've set it up so that's possible with RCP too. You might try having setThisRCP take a non-const reference to an RCP, so Boost.Python is forced to use an lvalue converter and hence a non-temporary RCP. Good luck! Jim From talljimbo at gmail.com Sat Feb 4 17:40:55 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 04 Feb 2012 11:40:55 -0500 Subject: [C++-sig] make_constructor and extending from python In-Reply-To: References: Message-ID: <4F2D5F97.3010305@gmail.com> On 02/03/2012 03:13 PM, Holger Brandsmeier wrote: > Dear list, > > how can I have a static member function that acts like a constructor > in a Wrapper, i.e. I have a class that I want to extend from python? > > I have a wrapper class around a class (PfemSpace) that has this static > member functions: > > static RCP create( ... ) > { > RCP ret(new PfemSpaceWrapperT( ... )); > > ret->setThisRCP(ret); > return ret; > } > > This method is exported via > .def("__init__", make_constructor(&ClassWrapperT::create ) ) > > This works without errors, but when I use this from python via > > class PfemSpaceStaticCond(PfemSpace): > def __init__(self, ...): > super(PfemSpaceStaticCond, self).__init__(...) > > def visit_createFemData(self): > print '## [PfemSpaceStaticCond_init] visit_createFemData' > > Here I call the constructor that I exported via make_constructor in > __init__(). Unfortunately the member function visit_createFemData that > I override in python never gets called. When I use a "real" > constructor instead of the one exported via make_constructor() then > the function visit_createFemData gets called. > > I assume the error is somewhere that with a real constructor > boost::python can call an inplace constructor, while with my > make_constructor it can't. Is there some way to have the needed > functionality with make_constructor? > > Note: today I came to the conclusion that I a read constructor instead > of a static member function is not an option for me due to the issues > I wrote in my previous mail "storing weak_ptr to this in classes > extended from python". That previous mail is actually from yesterday, > but it originally got rejected because I send it from a wrong email > address, so I'm sorry that you are receiving two mails from me today. > For a moment I was very happy that this "static constructor-like > function in the Wrapper class" would solve all my problems, but I am > still missing something. > I'm hoping my reply to your other email may give you a way forward, because I think there's a possibility that you've run into a real limitation of Boost.Python here. The only thing I can think of is to override __new__, and not override __init__; that might get the make_constructor version called. But I'm just guessing. make_constructor, useful as it is, just doesn't seem to really have the polish the rest of Boost.Python has, and I wouldn't be surprised if it just doesn't work here. After all, if you're wrapping a function that returns a smart pointer, there's no guarantee that contains an instance of the wrapper class needed to support Python-side polymorphism rather than just an instance of the C++ base class. Jim From talljimbo at gmail.com Sat Feb 4 17:46:52 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 04 Feb 2012 11:46:52 -0500 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: References: Message-ID: <4F2D60FC.8080005@gmail.com> On 02/04/2012 11:11 AM, Adam Preble wrote: > I am trying to run this on Win32, using Stackless Python 2.6, and Boost > 1.47. > > I am having a problem with passing a shared_ptr of a C++ object > implementing a C++ interface to a wrapper of a Python object > implementing a C++ interface. That's a mouthful, so I thought I'd > simplify the code. The thing is, I can't seem to get the wrapping > correct. If I can get this going, I can start adding complication to it > and see what the problem really is. > > Basically, I have an interface that works with itself for communicating > stuff. There is a C implementation of it, and a Python implementation > of it. I have to write a wrapper. The Python implementation can > communicate with instances of the C implementation, but not vice versa. > When the C implementation calls the Python one, I get: > > Traceback (most recent call last): > File "", line 1, in > TypeError: No to_python (by-value) converter found for C++ type: class > boost::shared_ptr > > The magic call is in the python script. > > Python script > http://pastebin.com/DJ65tKEW > > Header > http://pastebin.com/6BX8e6KA > > Source (Boost.Python declarations here) > http://pastebin.com/s5YC1hyC > > I have done similar wrappers like this before so I'm just befuddled > about why this time it hates me. The shared_ptrs are part of the larger > problem I'm trying to reproduce, so I want to incorporate them into this > test program. In the larger program, while doing a get_override() call > on a wrapper to an interface, I get a segfault inside the Python DLLs > while it tries to convert the shared_ptr. The last line I can see i > shared_ptr_to_python.hpp: > > return converter::registered > const&>::converters.to_python(&x); > > That's what I am eventually hoping to reproduce by shooting back and > forth all this stuff. > If I understand your intend correctly, I think your class_ definition needs to be: class_,noncopyable> (right now you have class_) Jim From johnny531 at gmail.com Sun Feb 5 01:43:37 2012 From: johnny531 at gmail.com (Johnny Graettinger) Date: Sat, 4 Feb 2012 19:43:37 -0500 Subject: [C++-sig] boost::python bindings for protobuf Message-ID: All, For those working with both protobuf and boost::python, I've written a protobuf plugin which generates boost::python wrappings for the protoc-generated C++ implementation. The rationale of the plugin (over, say, using the protoc experimental C++ python implementation) was a desire to directly wrap other C++ methods taking / returning protobuf message instances via boost::python. Presented in the hope that others find it useful: https://github.com/dademurphy/protobpl cheers, johnny -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.preble at gmail.com Sun Feb 5 02:23:53 2012 From: adam.preble at gmail.com (Adam Preble) Date: Sat, 4 Feb 2012 19:23:53 -0600 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: <4F2D60FC.8080005@gmail.com> References: <4F2D60FC.8080005@gmail.com> Message-ID: On Sat, Feb 4, 2012 at 10:46 AM, Jim Bosch wrote: > If I understand your intend correctly, I think your class_ definition > needs to be: > > class_,**noncopyable> > > (right now you have class_) > > > Something like this? class_, boost::noncopyable>("Communicatable") I get burnt by the compiler when I try that: d:\coding\boost_1_47_0\boost\python\object\pointer_holder.hpp(217): error C2664: 'CommunicatableWrapper::CommunicatableWrapper(const CommunicatableWrapper &)' : cannot convert parameter 1 from 'PyObject *' to 'const CommunicatableWrapper &' Reason: cannot convert from 'PyObject *' to 'const CommunicatableWrapper' No constructor could take the source type, or constructor overload resolution was ambiguous It sounds like if I had a constructor that would take a PyObject *, that would shut it up, but I have no idea what to make of such a thing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Sun Feb 5 03:48:47 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 04 Feb 2012 21:48:47 -0500 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: References: <4F2D60FC.8080005@gmail.com> Message-ID: <4F2DEE0F.8050501@gmail.com> On 02/04/2012 08:23 PM, Adam Preble wrote: > > > On Sat, Feb 4, 2012 at 10:46 AM, Jim Bosch > wrote: > > If I understand your intend correctly, I think your class_ > definition needs to be: > > class_,__noncopyable> > > (right now you have class_) > > > > Something like this? > > class_, > boost::noncopyable>("Communicatable") > > I get burnt by the compiler when I try that: > > d:\coding\boost_1_47_0\boost\python\object\pointer_holder.hpp(217): > error C2664: 'CommunicatableWrapper::CommunicatableWrapper(const > CommunicatableWrapper &)' : cannot convert parameter 1 from 'PyObject *' > to 'const CommunicatableWrapper &' > Reason: cannot convert from 'PyObject *' to 'const > CommunicatableWrapper' > No constructor could take the source type, or constructor > overload resolution was ambiguous > It sounds like if I had a constructor that would take a PyObject *, that > would shut it up, but I have no idea what to make of such a thing. > Oh, you're right. I was confusing the manual way of doing Python-side polymorphism (derived-class holders) with the more automatic (and better) way you're doing it (with wrapper). Your original code was fine in that regard. Anyhow, looking closer, here's what you need to do: - Put your code back to what it was before I told you to change it. - Add another line, after the class wrappers: register_ptr_to_python< boost::shared_ptr >(); That explicitly registers a shared_ptr converter for Communicatable. I'm not sure why Boost.Python doesn't automatically do that when you register converters for its derived classes (those are registered by putting the shared_pr types in the template parameters to class_). It might just be an oversight, or it might be that doing the base class registration automatically would cause problems in some contexts. Jim From adam.preble at gmail.com Sun Feb 5 06:56:37 2012 From: adam.preble at gmail.com (Adam Preble) Date: Sat, 4 Feb 2012 23:56:37 -0600 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: <4F2DEE0F.8050501@gmail.com> References: <4F2D60FC.8080005@gmail.com> <4F2DEE0F.8050501@gmail.com> Message-ID: On Sat, Feb 4, 2012 at 8:48 PM, Jim Bosch wrote: > Oh, you're right. I was confusing the manual way of doing Python-side > polymorphism (derived-class holders) with the more automatic (and better) > way you're doing it (with wrapper). Your original code was fine in that > regard. > > Anyhow, looking closer, here's what you need to do: > > - Put your code back to what it was before I told you to change it. > > - Add another line, after the class wrappers: > > register_ptr_to_python< boost::shared_ptr<**Communicatable> >(); > > That explicitly registers a shared_ptr converter for Communicatable. I'm > not sure why Boost.Python doesn't automatically do that when you register > converters for its derived classes (those are registered by putting the > shared_pr types in the template parameters to class_). It might just be an > oversight, or it might be that doing the base class registration > automatically would cause problems in some contexts. > > > The appears to do the trick. I'm wondering--how do you figure out stuff like that? That isn't the kind of thing that could have just occurred to me. This little test program works, but so far I haven't resolved the crash in my more complicated code that's coming a Python subclass of a C class trying to get a callback from Python instance of a C class . . . it's rough even explaining it. I'll have to review my code and make sure I'm doing that in all the right places. Either I get it working because of these little things here, or I'll be back with a modification of this source to show the problem with shared_ptr conversion that I was seeing. -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Sun Feb 5 17:15:19 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sun, 05 Feb 2012 11:15:19 -0500 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: References: <4F2D60FC.8080005@gmail.com> <4F2DEE0F.8050501@gmail.com> Message-ID: <4F2EAB17.205@gmail.com> On 02/05/2012 12:56 AM, Adam Preble wrote: > > On Sat, Feb 4, 2012 at 8:48 PM, Jim Bosch > wrote: > > Oh, you're right. I was confusing the manual way of doing > Python-side polymorphism (derived-class holders) with the more > automatic (and better) way you're doing it (with wrapper). Your > original code was fine in that regard. > > Anyhow, looking closer, here's what you need to do: > > - Put your code back to what it was before I told you to change it. > > - Add another line, after the class wrappers: > > register_ptr_to_python< boost::shared_ptr<__Communicatable> >(); > > That explicitly registers a shared_ptr converter for Communicatable. > I'm not sure why Boost.Python doesn't automatically do that when you > register converters for its derived classes (those are registered by > putting the shared_pr types in the template parameters to class_). > It might just be an oversight, or it might be that doing the base > class registration automatically would cause problems in some contexts. > > > The appears to do the trick. I'm wondering--how do you figure out stuff > like that? That isn't the kind of thing that could have just occurred > to me. > To be honest, it's probably just a result of having stared at the Boost.Python internals a lot while writing extensions for it. Ideally, that's not something every user would have to do - it's a great library, but the tutorial documentation does not go very deep, and the reference docs are only good if you know what you're looking for. So just keep mailing the list :) Another great way to learn more advance Boost.Python usage is to run Py++. Even if you don't use it to produce your final wrappers (I generally don't), looking at the output can teach you a lot of new tricks. The important bit of knowledge in this case is that putting a shared_ptr type in the template args to class_ does more than just register a to-python converter for shared_ptrs. It also causes new instances created in Python to be held in a shared_ptr. That's necessary in your case because you're using enable_shared_from_this, but usually it isn't if you just want to be able to return things by shared_ptr. So I usually use register_ptr_to_python on all my classes instead, and that's why I was already familiar with it. > This little test program works, but so far I haven't resolved the crash > in my more complicated code that's coming a Python subclass of a C class > trying to get a callback from Python instance of a C class . . . it's > rough even explaining it. I'll have to review my code and make sure I'm > doing that in all the right places. Either I get it working because of > these little things here, or I'll be back with a modification of this > source to show the problem with shared_ptr conversion that I was seeing. > Good luck! Jim From adam.preble at gmail.com Mon Feb 6 01:49:49 2012 From: adam.preble at gmail.com (Adam Preble) Date: Sun, 5 Feb 2012 18:49:49 -0600 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: <4F2EAB17.205@gmail.com> References: <4F2D60FC.8080005@gmail.com> <4F2DEE0F.8050501@gmail.com> <4F2EAB17.205@gmail.com> Message-ID: On Feb 5, 2012 10:15 AM, "Jim Bosch" wrote: > > On 02/05/2012 12:56 AM, Adam Preble wrote: >> >> >> On Sat, Feb 4, 2012 at 8:48 PM, Jim Bosch > > wrote: >> >> Oh, you're right. I was confusing the manual way of doing >> Python-side polymorphism (derived-class holders) with the more >> automatic (and better) way you're doing it (with wrapper). Your >> original code was fine in that regard. >> >> Anyhow, looking closer, here's what you need to do: >> >> - Put your code back to what it was before I told you to change it. >> >> - Add another line, after the class wrappers: >> >> register_ptr_to_python< boost::shared_ptr<__Communicatable> >(); >> >> >> That explicitly registers a shared_ptr converter for Communicatable. >> I'm not sure why Boost.Python doesn't automatically do that when you >> register converters for its derived classes (those are registered by >> putting the shared_pr types in the template parameters to class_). >> It might just be an oversight, or it might be that doing the base >> class registration automatically would cause problems in some contexts. >> >> >> The appears to do the trick. I'm wondering--how do you figure out stuff >> like that? That isn't the kind of thing that could have just occurred >> to me. >> > > To be honest, it's probably just a result of having stared at the Boost.Python internals a lot while writing extensions for it. Ideally, that's not something every user would have to do - it's a great library, but the tutorial documentation does not go very deep, and the reference docs are only good if you know what you're looking for. So just keep mailing the list :) > > Another great way to learn more advance Boost.Python usage is to run Py++. Even if you don't use it to produce your final wrappers (I generally don't), looking at the output can teach you a lot of new tricks. > I will have to look at it in the next few days, although I fear it may be an adventure on my real project. > The important bit of knowledge in this case is that putting a shared_ptr type in the template args to class_ does more than just register a to-python converter for shared_ptrs. It also causes new instances created in Python to be held in a shared_ptr. That's necessary in your case because you're using enable_shared_from_this, but usually it isn't if you just want to be able to return things by shared_ptr. So I usually use register_ptr_to_python on all my classes instead, and that's why I was already familiar with it. Jim, Would using register_ptr_to_python explicitly and consistently instead of the online shared_ptr declaration potentially eliminate some side-effects? I cannot yet isolate why my real app crashes in python when it tries to do an eval into one of my wrappers. -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulc.mnt at gmail.com Mon Feb 6 09:16:25 2012 From: paulc.mnt at gmail.com (Paul-Cristian Manta) Date: Mon, 6 Feb 2012 10:16:25 +0200 Subject: [C++-sig] How to define a Python metaclass with Boost.Python? Message-ID: The Python C API has the `PyObject *PyType_Type` object, which is equivalent to `type` in the interpreter. If I want to define a metaclass in C++, how can I set `type` as one of its bases in Boost.Python? Also, what other things should I take into consideration when defining a Python metaclass in C++? It'd be ideal if there was a Boost.Python solution to this. If not, a solution that uses the Python C API (or a combination of Boost and the C API) is good as well. -------------- next part -------------- An HTML attachment was scrubbed... URL: From ndbecker2 at gmail.com Mon Feb 6 17:33:12 2012 From: ndbecker2 at gmail.com (Neal Becker) Date: Mon, 06 Feb 2012 11:33:12 -0500 Subject: [C++-sig] boost::python handles boost::shared_ptr specially... Message-ID: Will std::shared_ptr just work also? From super24bitsound at hotmail.com Tue Feb 7 02:33:00 2012 From: super24bitsound at hotmail.com (Jay Riley) Date: Mon, 6 Feb 2012 20:33:00 -0500 Subject: [C++-sig] boost ref problem Message-ID: I'm trying to wrap a C++ class into python. I need to pass an enum value by reference through a virtual function. I'm doing it like so (non relevant code omitted): class BasicRMLScreenWrap : public BasicRMLScreen { public: bool HandleKeyPressedDefault(const sf::Uint32 time, const ::Input::InputModule* inputModule, ::Input::PlayerInput pinput, ::Input::InputAction& iaction) { return BasicRMLScreen::HandleKeyPressed(time, inputModule, pinput, iaction); } bool HandleKeyReleased(const sf::Uint32 time, const ::Input::InputModule* inputModule, ::Input::PlayerInput pinput, ::Input::InputAction& iaction) override { return call_method(self, "HandleKeyReleased", time, ptr(inputModule), pinput, boost::ref(iaction)); } private: PyObject* self; }; class_, boost::shared_ptr, boost::noncopyable >("BasicRMLScreen", init()) .def("HandleKeyPressed", &BasicRMLScreen::HandleKeyPressed, &BasicRMLScreenWrap::HandleKeyPressedDefault) ; the problem is that when I wrap it like this, I get the following error: Error 74 error C2664: 'boost::mpl::assertion_failed' : cannot convert parameter 1 from 'boost::mpl::failed ************boost::mpl::or_::* ***********' to 'boost::mpl::assert::type' if I switch boost::ref(iaction) to iaction it compiles fine, but I need to pass it by ref for correct behaviour. I'm not sure if this is a boost::ref problem or a boost python issue but was curious if someone might have an idea how to correct it. I can pass using boost:ref fine in the other places ive used it with call_method. Is this is an issues because it's an enum? Any help would be appreciated, thanks -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulc.mnt at gmail.com Tue Feb 7 07:35:11 2012 From: paulc.mnt at gmail.com (paulcmnt) Date: Mon, 6 Feb 2012 22:35:11 -0800 (PST) Subject: [C++-sig] How to define a Python metaclass with Boost.Python? Message-ID: <1328596511784-4363906.post@n4.nabble.com> The Python C API has the `PyObject *PyType_Type` object, which is equivalent to `type` in the interpreter. If I want to define a metaclass in C++, how can I set `type` as one of its bases in Boost.Python? Also, what other things should I take into consideration when defining a Python metaclass in C++? It'd be ideal if there was a Boost.Python solution to this. If not, a solution that uses the Python C API (or a combination of Boost and the C API) is good as well. (The version I'm using is Python 3.) -- View this message in context: http://boost.2283326.n4.nabble.com/How-to-define-a-Python-metaclass-with-Boost-Python-tp4363906p4363906.html Sent from the Python - c++-sig mailing list archive at Nabble.com. From talljimbo at gmail.com Tue Feb 7 17:11:46 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Tue, 07 Feb 2012 11:11:46 -0500 Subject: [C++-sig] boost ref problem In-Reply-To: References: Message-ID: <4F314D42.6020206@gmail.com> On 02/06/2012 08:33 PM, Jay Riley wrote: > > I'm trying to wrap a C++ class into python. I need to pass an enum value by reference through a virtual function. I'm doing it like so (non relevant code omitted): > > class BasicRMLScreenWrap : public BasicRMLScreen { public: bool HandleKeyPressedDefault(const sf::Uint32 time, const ::Input::InputModule* inputModule, ::Input::PlayerInput pinput, ::Input::InputAction& iaction) { return BasicRMLScreen::HandleKeyPressed(time, inputModule, pinput, iaction); } bool HandleKeyReleased(const sf::Uint32 time, const ::Input::InputModule* inputModule, ::Input::PlayerInput pinput, ::Input::InputAction& iaction) override { return call_method(self, "HandleKeyReleased", time, ptr(inputModule), pinput, boost::ref(iaction)); } private: PyObject* self; }; > class_, boost::shared_ptr, boost::noncopyable>("BasicRMLScreen", init()) .def("HandleKeyPressed",&BasicRMLScreen::HandleKeyPressed,&BasicRMLScreenWrap::HandleKeyPressedDefault) ; > the problem is that when I wrap it like this, I get the following error: > Error 74 error C2664: 'boost::mpl::assertion_failed' : cannot convert parameter 1 from 'boost::mpl::failed ************boost::mpl::or_::* ***********' to 'boost::mpl::assert::type' > > if I switch boost::ref(iaction) to iaction it compiles fine, but I need to pass it by ref for correct behaviour. I'm not sure if this is a boost::ref problem or a boost python issue but was curious if someone might have an idea how to correct it. I can pass using boost:ref fine in the other places ive used it with call_method. Is this is an issues because it's an enum? > Any help would be appreciated, I'm afraid you're run into a bit of an impossible problem. Boost.Python only defines rvalue from-python converters for enums, and this actually makes sense - like int, float, and str in Python, enums are immutable Python objects; when you set them, you're really just assigning the name of the variable to a new object. There's really no way to express modifying an argument in-place in Python when the argument is an immutable object. I think you'll manually have to transform the signature to return the new enum value in Python (i.e. make a derived class that implements the enum ref virtual member function by delegating to a new enum-returning virtual member function that you can wrap). Jim From talljimbo at gmail.com Tue Feb 7 17:14:54 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Tue, 07 Feb 2012 11:14:54 -0500 Subject: [C++-sig] How to define a Python metaclass with Boost.Python? In-Reply-To: <1328596511784-4363906.post@n4.nabble.com> References: <1328596511784-4363906.post@n4.nabble.com> Message-ID: <4F314DFE.2000704@gmail.com> On 02/07/2012 01:35 AM, paulcmnt wrote: > The Python C API has the `PyObject *PyType_Type` object, which is equivalent > to `type` in the interpreter. If I want to define a metaclass in C++, how > can I set `type` as one of its bases in Boost.Python? Also, what other > things should I take into consideration when defining a Python metaclass in > C++? It'd be ideal if there was a Boost.Python solution to this. If not, a > solution that uses the Python C API (or a combination of Boost and the C > API) is good as well. (The version I'm using is Python 3.) > I'm afraid this isn't really supported. All Boost.Python classes have to use a specific Boost.Python metaclass that's defined deep in the bowels of some Boost.Python source file, and there's really no way to override that at present. This might be a nice feature for the future, but I haven't thought too deeply about how to make it work or why one would want to do it. If you have a specific use case for a custom metaclass, I'd love to hear what it is. Jim From talljimbo at gmail.com Tue Feb 7 17:21:47 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Tue, 07 Feb 2012 11:21:47 -0500 Subject: [C++-sig] How do I make correct wrappers to interfaces while using shared_ptrs? In-Reply-To: References: <4F2D60FC.8080005@gmail.com> <4F2DEE0F.8050501@gmail.com> <4F2EAB17.205@gmail.com> Message-ID: <4F314F9B.7090504@gmail.com> On 02/05/2012 07:49 PM, Adam Preble wrote: > Would using register_ptr_to_python explicitly and consistently instead of > the online shared_ptr declaration potentially eliminate some side-effects? > I cannot yet isolate why my real app crashes in python when it tries to do > an eval into one of my wrappers. Overall, I doubt it. Usually putting shared_ptr in the class_ template params would allow you to do more fancy shared_ptr tricks, not less. And as I said before, it might be necessary if you use enable_shared_from_this. What your previous example showed, though, was that it might be useful to use register_ptr_to_python on every type you ever make a shared_ptr to in C++ *in addition* to putting shared_ptr in the class_ templates - you'll get some warnings in Python about redundant converters, and those will tell you which of those extra declarations it's safe to remove. Any that remain might help fix your problem. Jim From paulc.mnt at gmail.com Wed Feb 8 15:24:13 2012 From: paulc.mnt at gmail.com (Paul-Cristian Manta) Date: Wed, 8 Feb 2012 16:24:13 +0200 Subject: [C++-sig] How to define a Python metaclass with Boost.Python? Message-ID: > ---------- Forwarded message ---------- > From: Jim Bosch > To: Development of Python/C++ integration > Cc: > Date: Tue, 07 Feb 2012 11:14:54 -0500 > Subject: Re: [C++-sig] How to define a Python metaclass with Boost.Python? > On 02/07/2012 01:35 AM, paulcmnt wrote: > >> The Python C API has the `PyObject *PyType_Type` object, which is >> equivalent >> to `type` in the interpreter. If I want to define a metaclass in C++, how >> can I set `type` as one of its bases in Boost.Python? Also, what other >> things should I take into consideration when defining a Python metaclass >> in >> C++? It'd be ideal if there was a Boost.Python solution to this. If not, a >> solution that uses the Python C API (or a combination of Boost and the C >> API) is good as well. (The version I'm using is Python 3.) >> >> > I'm afraid this isn't really supported. All Boost.Python classes have to > use a specific Boost.Python metaclass that's defined deep in the bowels of > some Boost.Python source file, and there's really no way to override that > at present. > > This might be a nice feature for the future, but I haven't thought too > deeply about how to make it work or why one would want to do it. If you > have a specific use case for a custom metaclass, I'd love to hear what it > is. > > Jim > > In my application I have a base Event class and both C++ and Python inherit from this base and define new types of events. Clients can subscribe a certain function to a certain type of event. >>> events = EventDispatcher() >>> sub = events.subscribe(KeyboardEvent, on_keyboard_event) >>> sub.cancel() When an event is posted to the EventDispatcher (from either C++ or Python), all functions that are subscribed to that event, *or to a base of that event *, are called. Because of how the system is supposed to work, I need a common way to determine type and parents of an event at runtime, regardless of whether the event class was defined in C++ or Python; I do this with a custom EventTypeInfo class. The metaclass that I would like to define in C++ would add a __cpp_event_typeinfo attribute to each event class defined in Python, which I can then grab when I need it. *A possible solution.* I though a good enough way of defining a metaclass in Boost.Python would be to the equivalent of this code below. Do you think there are any safety problem with doing something like this, some compatibly problem that could arise with the other Boost.Python classes? >>> class Meta: ... pass ... >>> Meta = type(Meta.__name__, (type,), dict(Meta.__dict__)) -------------- next part -------------- An HTML attachment was scrubbed... URL: From rogeeff at gmail.com Fri Feb 10 22:57:49 2012 From: rogeeff at gmail.com (Gennadiy Rozental) Date: Fri, 10 Feb 2012 21:57:49 +0000 (UTC) Subject: [C++-sig] Crash at shutdown Message-ID: Hi, I need help with a crash I observe both on windows and Linux. I admit my use case is somewhat complicated, but that's reality. So, my problem consists form 3 components. Let's call them A, B, and C: A is Python *extension* library (dll/so). This is really just an infrastructure layer which helps with developing Python extensions like B+C. I am using Python 2.6 BTW. B is entry point into huge extension library which consists of multiple dlls/sos (one of them is C). This is also dll/so. C is library within my project, which is responsible for communication with Python *embedded* scripts. C is using Boost.Python to export some number of symbols into Python, but the issue occurs with just one as well. C is linked with Boost.Python library and Python interpreter Now imagine that I have a Python script which does essentially this: import A B=A.load('B') B.load('C') I run this script and observe a crash at Python shutdown. Here are more details about what going on in this script. Line 1 loads Python extension Line 2 loads library B using dlopen Line 3 loads library C using dlopen Object B is being released. This is done in two steps. First we invoke B.shutdown() which unloads C from memory using dlclose and next B is unloaded from memory using dlclose A is unloaded Python interpreter is shut down The crash occurs at the very last step in rather obscure place within Python interpreter shutdown routines. I found there are several "changes", which prevent the crash from happening: 1. I can stop exporting Boost.Python symbols in C 2. I can skip unloading C from B.shutdown() 3. I can link in C with B. This results in line 2 loading both B and C together. Line 3 does nothing. B.shutdown() does nothing and C is unloaded along with B when we call dlclose on B. At this point I am inclined to believe that something like this is happening: when I execute Boost.Python export statement in C it adds some records in Boost.Python and Python interpreter. When C is unloaded from memory somehow these records are not being cleaned up. By the time we get to clean this records C is already unloaded from memory and either Boost.Python or Python interpreter corrupt the memory. Any help is appreciated, Gennadiy From talljimbo at gmail.com Sat Feb 11 03:23:43 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Fri, 10 Feb 2012 21:23:43 -0500 Subject: [C++-sig] Crash at shutdown In-Reply-To: References: Message-ID: <4F35D12F.7010805@gmail.com> On 02/10/2012 04:57 PM, Gennadiy Rozental wrote: > Hi, > > I need help with a crash I observe both on windows and Linux. I admit my use > case is somewhat complicated, but that's reality. So, my problem consists form 3 > components. Let's call them A, B, and C: > > A is Python *extension* library (dll/so). This is really just an infrastructure > layer which helps with developing Python extensions like B+C. I am using Python > 2.6 BTW. > > B is entry point into huge extension library which consists of multiple dlls/sos > (one of them is C). This is also dll/so. > > C is library within my project, which is responsible for communication with > Python *embedded* scripts. C is using Boost.Python to export some number of > symbols into Python, but the issue occurs with just one as well. C is linked > with Boost.Python library and Python interpreter > > Now imagine that I have a Python script which does essentially this: > > import A > B=A.load('B') > B.load('C') > > I run this script and observe a crash at Python shutdown. Here are more details > about what going on in this script. > > Line 1 loads Python extension > Line 2 loads library B using dlopen > Line 3 loads library C using dlopen > Object B is being released. This is done in two steps. First we invoke > B.shutdown() which unloads C from memory using dlclose and next B is unloaded > from memory using dlclose > A is unloaded > Python interpreter is shut down > > The crash occurs at the very last step in rather obscure place within Python > interpreter shutdown routines. > > I found there are several "changes", which prevent the crash from happening: > > 1. I can stop exporting Boost.Python symbols in C > 2. I can skip unloading C from B.shutdown() > 3. I can link in C with B. This results in line 2 loading both B and C together. > Line 3 does nothing. B.shutdown() does nothing and C is unloaded along with B > when we call dlclose on B. > > At this point I am inclined to believe that something like this is happening: > > when I execute Boost.Python export statement in C it adds some records in > Boost.Python and Python interpreter. When C is unloaded from memory somehow > these records are not being cleaned up. By the time we get to clean this records > C is already unloaded from memory and either Boost.Python or Python interpreter > corrupt the memory. > This scenario sounds very possible, and a good candidate for those "records" is the Boost.Python converter registry. That's supposed to be a global registry (in the boost_python shared library) that's shared by all modules, and it mostly works quite well when the only dlopens involved are the ones the Python interpreter uses when importing modules. I could definitely imagine it being corrupted by doing a dlclose on the shared library that exported some classes using Boost.Python. I'm pretty sure there's no programmatic way to remove something from the registry, and to add such a feature you'd have to modify the Boost.Python sources and recompile the shared library. If you're willing to do that, that might be a way out. If it's at all possible, I think the safest bet would be to refactor things so that everything that gets exported to Python happens within a separate module that would be imported by the Python scripts, so you only rely on Python's own dlopen calls when it involves Boost.Python wrappers. If that's not feasible, you might try putting the wrapper code in a function in a library that never gets unloaded, even if that function is called by some library that may be unloaded. Good luck - sounds like a hard problem to solve! Jim Bosch From talljimbo at gmail.com Sat Feb 11 06:09:24 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 11 Feb 2012 00:09:24 -0500 Subject: [C++-sig] How to define a Python metaclass with Boost.Python? In-Reply-To: References: Message-ID: <4F35F804.4070802@gmail.com> Sorry about the long delay... On 02/08/2012 09:24 AM, Paul-Cristian Manta wrote: > In my application I have a base Event class and both C++ and Python > inherit from this base and define new types of events. Clients can > subscribe a certain function to a certain type of event. > > >>> events = EventDispatcher() > >>> sub = events.subscribe(KeyboardEvent, on_keyboard_event) > >>> sub.cancel() > > When an event is posted to theEventDispatcher (from either C++ or > Python), all functions that are subscribed to that event, /or to a base > of that event/, are called. > > Because of how the system is supposed to work, I need a common way to > determine type and parents of an event at runtime, regardless of whether > the event class was defined in C++ or Python; I do this with a custom > EventTypeInfo class. > > The metaclass that I would like to define in C++ would add a > __cpp_event_typeinfo attribute to each event class defined in Python, > which I can then grab when I need it. > > *A possible solution.* I though a good enough way of defining a > metaclass in Boost.Python would be to the equivalent of this code below. > Do you think there are any safety problem with doing something like > this, some compatibly problem that could arise with the other > Boost.Python classes? > > >>> class Meta: > ... pass > ... > >>> Meta = type(Meta.__name__, (type,), dict(Meta.__dict__)) > This is indeed a perfectly valid way to make a metaclass, and it should definitely work to do the same thing in C++ using the C API and/or Boost.Python. The trouble is using that metaclass as the type of a Boost.Python-wrapped C++ class...that's what I think is basically impossible. Of course, if you just want to inject some attributes into a wrapped class, you can do that in Boost.Python without a metaclass. You'd write a function that takes a boost::python::object and adds the attributes you want to it. Calling that function with a boost::python::class_ object would then add the attribute, just like a metaclass would. Jim From talljimbo at gmail.com Sat Feb 11 06:26:28 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 11 Feb 2012 00:26:28 -0500 Subject: [C++-sig] boost::python handles boost::shared_ptr specially... In-Reply-To: References: Message-ID: <4F35FC04.4060504@gmail.com> On 02/06/2012 11:33 AM, Neal Becker wrote: > Will std::shared_ptr just work also? Just tried the experiment, albeit with slightly old versions. As of gcc 4.4 (with -std=gnu++0x) and Boost 1.47, it looks like Boost.Python doesn't even recognize std::shared_ptr as any kind of smart pointer, let alone an equivalent to boost::shared_ptr. That's definitely going to need fixing in the near future. Jim From ndbecker2 at gmail.com Sat Feb 11 15:03:34 2012 From: ndbecker2 at gmail.com (Neal Becker) Date: Sat, 11 Feb 2012 09:03:34 -0500 Subject: [C++-sig] boost::python handles boost::shared_ptr specially... References: <4F35FC04.4060504@gmail.com> Message-ID: Jim Bosch wrote: > On 02/06/2012 11:33 AM, Neal Becker wrote: >> Will std::shared_ptr just work also? > > Just tried the experiment, albeit with slightly old versions. > > As of gcc 4.4 (with -std=gnu++0x) and Boost 1.47, it looks like > Boost.Python doesn't even recognize std::shared_ptr as any kind of smart > pointer, let alone an equivalent to boost::shared_ptr. > > That's definitely going to need fixing in the near future. > > Jim Thanks: https://svn.boost.org/trac/boost/ticket/6545 From s_sourceforge at nedprod.com Sat Feb 11 14:59:13 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Sat, 11 Feb 2012 13:59:13 -0000 Subject: [C++-sig] Crash at shutdown In-Reply-To: <4F35D12F.7010805@gmail.com> References: , <4F35D12F.7010805@gmail.com> Message-ID: <4F367431.5724.A52918B1@s_sourceforge.nedprod.com> On 10 Feb 2012 at 21:23, Jim Bosch wrote: > > when I execute Boost.Python export statement in C it adds some records in > > Boost.Python and Python interpreter. When C is unloaded from memory somehow > > these records are not being cleaned up. By the time we get to clean this records > > C is already unloaded from memory and either Boost.Python or Python interpreter > > corrupt the memory. > > I'm pretty sure there's no programmatic way to remove something from the > registry, and to add such a feature you'd have to modify the > Boost.Python sources and recompile the shared library. If you're > willing to do that, that might be a way out. I was about to say the same as Jim, except to add that this is really another example of why BPL lacks Py_Finalize() support if I remember correctly. BPL can set itself up, but it's a one way action - it cannot unwind itself. > If it's at all possible, I think the safest bet would be to refactor > things so that everything that gets exported to Python happens within a > separate module that would be imported by the Python scripts, so you > only rely on Python's own dlopen calls when it involves Boost.Python > wrappers. If that's not feasible, you might try putting the wrapper > code in a function in a library that never gets unloaded, even if that > function is called by some library that may be unloaded. The other thing to try is to force an immediate exit without unwinding the DLL list :) e.g. TerminateProcess(self). Obviously, do this after all buffers and such have been flushed, but just before DLL unload. The other thing, on Windows, is to manually hack increment the DLL reference count to ensure the DLL never gets kicked out :) This works well for this type of situation. Sometimes when working with other people's broken libraries it's just easiest. BTW I agree that code which cannot shut itself down cleanly is broken and its authors should hang their heads appropriately and treat it as a bug to be fixed rather than a feature to be added. That said, it can be hard to anticipate every possible shutdown use case. I've certainly been wowed by some user reports regarding my own libraries coming in at times. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From paulc.mnt at gmail.com Sat Feb 11 22:15:32 2012 From: paulc.mnt at gmail.com (Paul-Cristian Manta) Date: Sat, 11 Feb 2012 23:15:32 +0200 Subject: [C++-sig] How to define a Python metaclass with Boost.Python? Message-ID: > > > This is indeed a perfectly valid way to make a metaclass, and it should > definitely work to do the same thing in C++ using the C API and/or > Boost.Python. > > The trouble is using that metaclass as the type of a Boost.Python-wrapped > C++ class...that's what I think is basically impossible. > > Of course, if you just want to inject some attributes into a wrapped > class, you can do that in Boost.Python without a metaclass. You'd write a > function that takes a boost::python::object and adds the attributes you > want to it. Calling that function with a boost::python::class_ object > would then add the attribute, just like a metaclass would. > > > Jim > > I don't think it's impossible to set a metaclass of another class with Boost.Python. The trick is actually pretty similar to the one above. >>> class Foo: ... pass ... >>> Foo = FooMeta(Foo.__name__, Foo.__bases__, dict(Foo.__dict__)) You first define the class in Boost.Python as you normally would, then do the equivalent of the code above. This should work, unless, of course, there's some subtle detail I'm not aware of. -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Sun Feb 12 01:55:15 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 11 Feb 2012 19:55:15 -0500 Subject: [C++-sig] How to define a Python metaclass with Boost.Python? In-Reply-To: References: Message-ID: <4F370DF3.1@gmail.com> On 02/11/2012 04:15 PM, Paul-Cristian Manta wrote: > > I don't think it's impossible to set a metaclass of another class with > Boost.Python. The trick is actually pretty similar to the one above. > > >>> class Foo: > ... pass > ... > >>> Foo = FooMeta(Foo.__name__, Foo.__bases__, dict(Foo.__dict__)) > > You first define the class in Boost.Python as you normally would, then > do the equivalent of the code above. This should work, unless, of > course, there's some subtle detail I'm not aware of. > Ah, yes. I see what you're talking about now. You'll probably need to have FooMeta inherit from the Boost.Python metaclass and delegate to its __call__ to make sure the Boost.Python metaclass gets to do its magic too. But I think that ought to be enough to make it work. Good thinking - glad you didn't get discouraged by my pessimism! Jim From eudoxos at arcig.cz Sun Feb 12 18:49:21 2012 From: eudoxos at arcig.cz (=?UTF-8?B?VsOhY2xhdiDFoG1pbGF1ZXI=?=) Date: Sun, 12 Feb 2012 18:49:21 +0100 Subject: [C++-sig] from-python converter for aligned class Message-ID: <4F37FBA1.5030505@arcig.cz> I wrote a custom to-python converter for an aligned struct (It is a 128-bit aligned vector type from the http://eigen.tuxfamily.org library). I followed http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/faq.html#custom_string), the converter looks like this: class AlignedType{ /*... */} __attribute__((aligned(16))); struct converter{ converter(){ py::converter::registry::push_back(&convertible,&construct,py::type_id()); static void* convertible(PyObject* obj_ptr){ /*...*/ } static void construct(PyObject* obj_ptr, py::converter::rvalue_from_python_stage1_data* data){ void* storage=((py::converter::rvalue_from_python_storage*)(data))->storage.bytes; // !! this creates AlignedType instance at a possibly unaligned address !! new (storage) AlignedType; data->convertible=storage; } }; I am getting crashes due to mis-alignment of the resulting object, because it is created at a possible unaligned address. Is there a way around this? Can I allocate another chunk of memory, or enforce the alignment before it is allocated? Best regards, Vaclav From eudoxos at arcig.cz Sun Feb 12 19:01:16 2012 From: eudoxos at arcig.cz (=?utf-8?b?VsOhY2xhdg==?= =?utf-8?b?xaBtaWxhdWVy?=) Date: Sun, 12 Feb 2012 18:01:16 +0000 (UTC) Subject: [C++-sig] =?utf-8?q?boost=3A=3Apython_handles_boost=3A=3Ashared?= =?utf-8?b?X3B0ciBzcGVjaWFsbHkuLi4=?= References: Message-ID: > Will std::shared_ptr just work also? I asked this question some time ago here: http://stackoverflow.com/questions/6568952/c0x-stdshared-ptr-vs-boostshared-ptr To summarize: you need to define free get_pointer functions for T=your shared_ptr type and it will work. I eventyallu did not go that way because boost::serialization (which I also needed) will not work with std::shared_ptr. From eudoxos at arcig.cz Sun Feb 12 19:07:42 2012 From: eudoxos at arcig.cz (=?utf-8?b?VsOhY2xhdg==?= =?utf-8?b?xaBtaWxhdWVy?=) Date: Sun, 12 Feb 2012 18:07:42 +0000 (UTC) Subject: [C++-sig] Boost converter for passing std::vector object by reference into a function?? References: <3D98DFAFDAF0DF4AB6643AF7394426720CFDA290@nasanexd02b.na.qualcomm.com> <4F1AF197.20909@astro.princeton.edu> Message-ID: > One could imagine writing a fancy converter that would allow the first > form by making a temporary std::vector, passing that to the C++ > function, and then copying the elements in the vector back into the > Python list. But that's potentially a very expensive sequence of > operations, and it's actually impossible to do that through a registered > converter in Boost.Python. In some cases, such a converter makes sense, I wrote one (see http://bazaar.launchpad.net/~eudoxos/+junk/tr2/view/head:/py/wrapper/customConverters.cpp#L142, then its specializations for various types below). Where std::vector arg is expected, it will check whether the python object is a sequence and then convert is to the c++ vector. Of course the objects are being copied, not referenced, unless those objects are shared_ptr's, in which case you get reference semantics you asked for. Cheers, vaclav From talljimbo at gmail.com Sun Feb 12 19:25:19 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sun, 12 Feb 2012 13:25:19 -0500 Subject: [C++-sig] from-python converter for aligned class In-Reply-To: <4F37FBA1.5030505@arcig.cz> References: <4F37FBA1.5030505@arcig.cz> Message-ID: <4F38040F.1020605@gmail.com> On 02/12/2012 12:49 PM, V?clav ?milauer wrote: > I wrote a custom to-python converter for an aligned struct (It is a > 128-bit aligned vector type from the http://eigen.tuxfamily.org > library). I followed > http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/faq.html#custom_string), > the converter looks like this: > > class AlignedType{ /*... */} __attribute__((aligned(16))); > struct converter{ > converter(){ > py::converter::registry::push_back(&convertible,&construct,py::type_id()); > > static void* convertible(PyObject* obj_ptr){ /*...*/ } > static void construct(PyObject* obj_ptr, > py::converter::rvalue_from_python_stage1_data* data){ > void* > storage=((py::converter::rvalue_from_python_storage*)(data))->storage.bytes; > > // !! this creates AlignedType instance at a possibly unaligned address !! > new (storage) AlignedType; > data->convertible=storage; > } > }; > > I am getting crashes due to mis-alignment of the resulting object, > because it is created at a possible unaligned address. Is there a way > around this? Can I allocate another chunk of memory, or enforce the > alignment before it is allocated? > Hmm. You might be able to make this work with another class that holds the AlignedType and has an implicit conversion to it: struct AlignedHolder { operator AlignedType const & () const { return *p; } std::auto_ptr p; }; You'd then write an rvalue converter for AlignedHolder, but register it with type_id(). Boost.Python will see the type_id for AlignedType and try to use it when you have an "AlignedType const &" argument, but it will try to pass the AlignedHolder object to your function...at which point the implicit conversion will kick in. I haven't tested this, and it involves really tricking Boost.Python, but I think it might work. Jim From avibahra at googlemail.com Sun Feb 12 19:25:45 2012 From: avibahra at googlemail.com (Avi Bahra) Date: Sun, 12 Feb 2012 18:25:45 +0000 Subject: [C++-sig] boost python & context manager/with statement Message-ID: Does boost python support context management using the with statement? In my case I need to manage __enter__/__exit__ on the c++ side. Could not find any examples for this in the documentation. Any help appreciated. Best regards, Ta, ? ? Avi From talljimbo at gmail.com Sun Feb 12 19:31:49 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sun, 12 Feb 2012 13:31:49 -0500 Subject: [C++-sig] boost::python handles boost::shared_ptr specially... In-Reply-To: References: Message-ID: <4F380595.5060707@gmail.com> On 02/12/2012 01:01 PM, V?clav?milauer wrote: >> Will std::shared_ptr just work also? > I asked this question some time ago here: > http://stackoverflow.com/questions/6568952/c0x-stdshared-ptr-vs-boostshared-ptr > > To summarize: you need to define free get_pointer functions for T=your > shared_ptr type and it will work. I eventyallu did not go that way because > boost::serialization (which I also needed) will not work with std::shared_ptr. > This would be a good start; it would make the std::shared_ptr support at least as good as, say, std::auto_ptr. It wouldn't be quite enough in some cases, though, because Boost.Python does a lot of special things for boost::shared_ptr, related to fact that it can have custom deleters. In particular, if you have a Boost.Python-wrapped C++ object that is held *by-value* inside a PyObject*, you can actually pass that to a C++ function by boost::shared_ptr, because Boost.Python creates that shared_ptr on the fly with a custom deleter that holds the PyObject*. That's should also be possible with std::shared_ptr, and Boost.Python would need to be told about that too. Jim From guillaume.carbonneau at gmail.com Sun Feb 12 19:34:15 2012 From: guillaume.carbonneau at gmail.com (Guillaume Carbonneau) Date: Sun, 12 Feb 2012 10:34:15 -0800 Subject: [C++-sig] boost python and boost/thread.hpp Message-ID: Hi, I'm having compilation errors as soon as I give my classes member variables that come from the boost thread library. Wrapping those in shared_ptr seems to work. Any idea why? Is there a special wrapping directive I should give it to? #include #include #include struct Hello { Hello(){} bool run(int argc, char **argv, char **envp); std::string hello_string_; std::vector hello_vector_ ; size_t hello_num_; boost::shared_ptr hello_shared_ptr_string_; bool hello_bool_; boost::shared_ptr condition_variable_shared_ptr_; // does not compile //boost::condition_variable condition_variable_; }; char const* greet() { return "hello, world"; } BOOST_PYTHON_MODULE(hello) { using namespace boost::python; def("greet", greet); class_("Hello"); } $ make g++ -c -o hello.o -I/System/Library/Frameworks/Python.framework/Versions/2.6/include/python2.6/ -Wall -O3 -fPIC -g hello.cpp /usr/local/include/boost/python/object/value_holder.hpp: In constructor ?boost::python::objects::value_holder::value_holder(PyObject*, A0) [with A0 = boost::reference_wrapper, Value = Hello]?: /usr/local/include/boost/python/object/make_instance.hpp:71: instantiated from ?static Holder* boost::python::objects::make_instance::construct(void*, PyObject*, boost::reference_wrapper) [with T = Hello, Holder = boost::python::objects::value_holder]? /usr/local/include/boost/python/object/make_instance.hpp:45: instantiated from ?static PyObject* boost::python::objects::make_instance_impl::execute(Arg&) [with Arg = const boost::reference_wrapper, T = Hello, Holder = boost::python::objects::value_holder, Derived = boost::python::objects::make_instance >]? /usr/local/include/boost/python/object/class_wrapper.hpp:29: instantiated from ?static PyObject* boost::python::objects::class_cref_wrapper::convert(const Src&) [with Src = Hello, MakeInstance = boost::python::objects::make_instance >]? /usr/local/include/boost/python/converter/as_to_python_function.hpp:27: instantiated from ?static PyObject* boost::python::converter::as_to_python_function::convert(const void*) [with T = Hello, ToPython = boost::python::objects::class_cref_wrapper > >]? /usr/local/include/boost/python/to_python_converter.hpp:87: instantiated from ?boost::python::to_python_converter::to_python_converter() [with T = Hello, Conversion = boost::python::objects::class_cref_wrapper > >, bool has_get_pytype = true]? /usr/local/include/boost/python/object/class_wrapper.hpp:26: instantiated from ?static void boost::python::objects::class_metadata::maybe_register_class_to_python(T2*, mpl_::false_) [with T2 = Hello, T = Hello, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]? /usr/local/include/boost/python/object/class_metadata.hpp:229: instantiated from ?static void boost::python::objects::class_metadata::register_aux2(T2*, Callback) [with T2 = Hello, Callback = boost::integral_constant, T = Hello, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]? /usr/local/include/boost/python/object/class_metadata.hpp:219: instantiated from ?static void boost::python::objects::class_metadata::register_aux(void*) [with T = Hello, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]? /usr/local/include/boost/python/object/class_metadata.hpp:205: instantiated from ?static void boost::python::objects::class_metadata::register_() [with T = Hello, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]? /usr/local/include/boost/python/class.hpp:496: instantiated from ?void boost::python::class_::initialize(const DefVisitor&) [with DefVisitor = boost::python::init, W = Hello, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]? /usr/local/include/boost/python/class.hpp:629: instantiated from ?boost::python::class_::class_(const char*, const char*) [with W = Hello, X1 = boost::python::detail::not_specified, X2 = boost::python::detail::not_specified, X3 = boost::python::detail::not_specified]? hello.cpp:32: instantiated from here /usr/local/include/boost/python/object/value_holder.hpp:137: error: no matching function for call to ?Hello::Hello(const Hello&)? hello.cpp:7: note: candidates are: Hello::Hello() hello.cpp:6: note: Hello::Hello(Hello&) make: *** [hello.o] Error 1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Sun Feb 12 19:43:31 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sun, 12 Feb 2012 13:43:31 -0500 Subject: [C++-sig] boost python & context manager/with statement In-Reply-To: References: Message-ID: <4F380853.6030709@gmail.com> On 02/12/2012 01:25 PM, Avi Bahra wrote: > Does boost python support context management using the with > statement? In my case I need to manage __enter__/__exit__ on > the c++ side. Could not find any examples for this in the > documentation. > Do you mean that you want to create an object that *has* __enter__ and __exit__ methods in C++, or that you want to write something like a "with" statement in C++? If it's the former, you can do that pretty easily - just call your methods "__enter__" and "__exit__" and give them the right signatures, and Python will use them quite happily. If it's the latter, I'm afraid there's no syntactic sugar for it. It's probably easiest to fake it by calling __enter__ and __exit__ and wrapping it all in a C++ try/catch block (if you raise a Python exception, it gets thrown in C++ as boost::python::error_already_set). But C++ itself doesn't really have a context management syntax, so there isn't really a good way to simulate the Python one. HTH Jim From stefan at seefeld.name Sun Feb 12 19:45:07 2012 From: stefan at seefeld.name (Stefan Seefeld) Date: Sun, 12 Feb 2012 13:45:07 -0500 Subject: [C++-sig] boost python and boost/thread.hpp In-Reply-To: References: Message-ID: <4F3808B3.6060005@seefeld.name> On 02/12/2012 01:34 PM, Guillaume Carbonneau wrote: > Hi, > > I'm having compilation errors as soon as I give my classes member > variables that come from the boost thread library. > Wrapping those in shared_ptr seems to work. By default, boost.python treats objects as copyable. That doesn't work if some of your members aren't copyable themselves. You need to figure out the semantics of your struct 'Hello'. Do you want it to be copyable, than you need to change the definition of the non-copyable members, such as you have done. However, you could as well make the wrapped type non-copyable, in which case that's not necessary. > > Any idea why? Is there a special wrapping directive I should give it to? See http://www.boost.org/doc/libs/1_48_0/libs/python/doc/v2/class.html Stefan -- ...ich hab' noch einen Koffer in Berlin... From talljimbo at gmail.com Sun Feb 12 19:56:12 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sun, 12 Feb 2012 13:56:12 -0500 Subject: [C++-sig] boost python and boost/thread.hpp In-Reply-To: References: Message-ID: <4F380B4C.8030000@gmail.com> On 02/12/2012 01:34 PM, Guillaume Carbonneau wrote: > Hi, > > I'm having compilation errors as soon as I give my classes member > variables that come from the boost thread library. > Wrapping those in shared_ptr seems to work. > > Any idea why? Is there a special wrapping directive I should give it to? > The problem is that boost::condition_variable doesn't have a copy constructor. When you declare a class with Boost.Python, it assumes it can be returned by value (which requires a copy constructor). The boost::condition_variable data member prevents the compiler from generating an implicit copy constructor. You can solve this by instead using: class_("Hello"); That tells Boost.Python not to assume this class can be returned by value. If it makes sense to define an explicit copy constructor for your class, that will work too. Jim From adam.preble at gmail.com Mon Feb 13 05:47:00 2012 From: adam.preble at gmail.com (Adam Preble) Date: Sun, 12 Feb 2012 22:47:00 -0600 Subject: [C++-sig] Runtime errors after trying to use debug Python build with Boost.Python Message-ID: I am trying to use Python with debug symbols so I can peak into it when I get runtime errors related to Boost.Python wrappers. I can't seem to get this to work. I think this is more a building thing than anything else so I figure I'd share what I've done: 1. Acquire the source for Stackless Python 2.6.5. Mind you, this release has been working for me standalone. 2. Build Stackless Python, or at least the "relevant parts." I don't have all the dependencies and according to the instructions, I should find what I need within in the python and pythoncore projects. I'm building on Windows, using the PcBuild project. 3. Copy over python.exe, python26.lib, and the python_d.dll to my Python26 directory. The build generates everything as python_d.exe, python26_d.lib, and all that. I have them all named instead as just python.exe, and python26.exe; I keep python_d.dll since it seems to look for that specifically at runtime. 4. I rebuild Boost.Python. I make sure to specifically overwrite: boost_python_vc100-mt-gd-1_47.lib boost_python_vc100-mt-gd-1_47.dll libboost_python_vc100-mt-gd-1_47.lib 5. I rebuild my project, and launch it. Along inside Py_Initialize I hit an assert: Assertion failed: (op->_ob_prev == NULL) == (op->_ob_next == NULL), file ..\Objects\object.c, line 65 I figure the assert doesn't mean anything to anybody, so I hoped instead you could see my procedure and point out where I botched something. FWIW, the debug python.exe otherwise seems to be working fine by itself. I can start it up and run some Python scripts through it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eudoxos at arcig.cz Mon Feb 13 11:07:18 2012 From: eudoxos at arcig.cz (=?utf-8?b?VsOhY2xhdg==?= =?utf-8?b?xaBtaWxhdWVy?=) Date: Mon, 13 Feb 2012 10:07:18 +0000 (UTC) Subject: [C++-sig] constructing aligned instance in python References: <4F37FBA1.5030505@arcig.cz> <4F38040F.1020605@gmail.com> Message-ID: > Hmm. You might be able to make this work with another class that holds > the AlignedType and has an implicit conversion to it: Nice trick, it surprisingly compiled. I found meanwhile though that the problem is more general: whenever a new instance of AlignedType is created from python, it will be possibly mis-aligned, because the Python allocator does not care about alignment: it simply provides a chunk of memory based on PyTypeObject.{tp_basicsize,tp_itemsize} for the type being instantiated. A possible workaround (though quite laborious) is to use the overloaded operator new ot AlignedType, which aligns properly, and write all ctors of the wrapped class such that pointer (manage_new_object, I assume?) is returned: namespace py=boost::python; static AlignedType* AlignedType_new(){ return new AlignedType; } static AlignedType* AlignedType_copy(const AlignedType& other){ return new AlignedType(other); } py::class_("AlignedType",boost::python::no_init) .def("__init__",&AlignedType_new) .def("__init__",&AlignedType_copy) ; Isn't that insane? Is there a better way? Cheers, v. From Dirk.Freyer at sicat.com Tue Feb 14 17:46:47 2012 From: Dirk.Freyer at sicat.com (Freyer Dirk (siCAT)) Date: Tue, 14 Feb 2012 16:46:47 +0000 Subject: [C++-sig] python32 64bit error LNK2001 Message-ID: Hi, I try the Boost.Python quickstart with Win7, Python32 64bit, Visual Studio 2010. But get an error like LNK2001. --------------------------------------------------------------------- msvc.link.dll ..\..\..\..\bin.v2\libs\python\build\msvc-10.0\debug\threading-multi\boost_python-vc100-mt-gd-1_48.dll Creating library ..\..\..\..\bin.v2\libs\python\build\msvc-10.0\debug\threading-multi\boost_python-vc100-mt-gd-1_48.lib an d object ..\..\..\..\bin.v2\libs\python\build\msvc-10.0\debug\threading-multi\boost_python-vc100-mt-gd-1_48.exp function.obj : error LNK2001: unresolved external symbol __imp__PyErr_Format Have someone any experience with 64bit python and boost? Can this work? I attached the full output file. Thanks Dirk -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: out.txt URL: From olivas at icecube.umd.edu Tue Feb 14 19:56:22 2012 From: olivas at icecube.umd.edu (Alex Olivas) Date: Tue, 14 Feb 2012 11:56:22 -0700 Subject: [C++-sig] Wrapping classes that inherit from classes already wrapped Message-ID: <4F3AAE56.2020306@icecube.umd.edu> Hey All, I was working on a project a few months ago where I was wrapping a lib ( using Boost.Python ) that inherited from classes that were already wrapped ( using SIP ). Boost.Python however couldn't find the base classes. I hacked something up ( that I'm not very proud of ) in class.cpp : type_handle get_class(type_info id) { type_handle result(query_class(id)); if (result.get() == 0) { PyObject* globals = PyEval_GetGlobals() ; PyObject* key = object( id.name() ).ptr() ; PyObject* obj = PyDict_GetItem( globals, key ); if( obj ){ return type_handle( obj->ob_type ); }else{ object report("extension class wrapper for base class "); report = report + id.name() + " has not been created yet"; PyErr_SetObject(PyExc_RuntimeError, report.ptr()); throw_error_already_set(); } } return result; } Surely there has to be a better way. Does anyone have a better solution than this? This was also followed by metaclass and layout conflicts, but that's another issue I think. To give a little background I was trying to wrap a lib that inherits from Qt and I really would prefer to use the bindings from PyQt rather than re-wrap ( which also seems more difficult than I thought it would be ). Thanks, Alex. -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.preble at gmail.com Wed Feb 15 16:38:38 2012 From: adam.preble at gmail.com (Adam Preble) Date: Wed, 15 Feb 2012 09:38:38 -0600 Subject: [C++-sig] Building Boost.Python against a debug build of Python Message-ID: I previously posted about runtime oddness when switching to a debug build of python. I figured I needed to rebuild Boost.Python. The build process is fixated on using the normal python.exe and python.lib paths, rather than python_d.exe and python_d.lib. I have put those files in my Python26 directory, and then tried to get fancy by renaming them to their release names instead. It gets mad at me . . . Creating library bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture-x86\debug-store-database\debug-symbols-on\python-debugging-on\threading-multi\boost_python-vc100-mt-y-1_47.lib and object bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture-x86\debug-store-database\debug-symbols-on\python-debugging-on\threading-multi\boost_python-vc100-mt-y-1_47.exp module.obj : error LNK2019: unresolved external symbol __imp__Py_InitModule4 referenced in function "struct _object * __cdecl boost::python::detail::init_module(char const *,void (__cdecl*)(void))" (?init_module at detail@python at boost@@YAPAU_object@@PBDP6AXXZ at Z) bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture-x86\debug-store-database\debug-symbols-on\python-debugging-on\threading-multi\boost_python-vc100-mt-y-1_47.dll : fatal error LNK1120: 1 unresolved externals I'm thinking that isn't what I'm supposed to do anyways. Is there something I need to do in my build string? This is how I would do a full build. Note all the baggage I added for building Python: [FWIW I'm building Boost 1.47 with VS2010's toolchain] set INCLUDE=D:\Python26\include;D:\Python26\include\Stackless b2 --toolset=msvc-10.0 --build-type=complete stage b2 --toolset=msvc-10.0 --build-type=complete architecture=x86 address-model=32 stage b2 --toolset=msvc-10.0 --with-python --build-type=complete --debug-configuration architecture=x86 address-model=32 stage debug-symbols=on debug-store=database python-debugging=on -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Fri Feb 17 23:15:19 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Fri, 17 Feb 2012 17:15:19 -0500 Subject: [C++-sig] Building Boost.Python against a debug build of Python In-Reply-To: References: Message-ID: <4F3ED177.3020604@gmail.com> If you haven't already, you may need to take this to a Boost.Build list; I avoid Boost's build system whenever possible, and I know even less about using it on Windows. Sorry! Jim On 02/15/2012 10:38 AM, Adam Preble wrote: > I previously posted about runtime oddness when switching to a debug build > of python. I figured I needed to rebuild Boost.Python. The build process > is fixated on using the normal python.exe and python.lib paths, rather than > python_d.exe and python_d.lib. I have put those files in my Python26 > directory, and then tried to get fancy by renaming them to their release > names instead. It gets mad at me . . . > > Creating library > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture-x86\debug-store-database\debug-symbols-on\python-debugging-on\threading-multi\boost_python-vc100-mt-y-1_47.lib > and object > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture-x86\debug-store-database\debug-symbols-on\python-debugging-on\threading-multi\boost_python-vc100-mt-y-1_47.exp > module.obj : error LNK2019: unresolved external symbol > __imp__Py_InitModule4 referenced in function "struct _object * __cdecl > boost::python::detail::init_module(char const *,void (__cdecl*)(void))" > (?init_module at detail@python at boost@@YAPAU_object@@PBDP6AXXZ at Z) > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture-x86\debug-store-database\debug-symbols-on\python-debugging-on\threading-multi\boost_python-vc100-mt-y-1_47.dll > : fatal error LNK1120: 1 unresolved externals > > I'm thinking that isn't what I'm supposed to do anyways. Is there > something I need to do in my build string? This is how I would do a full > build. Note all the baggage I added for building Python: > > [FWIW I'm building Boost 1.47 with VS2010's toolchain] > > set INCLUDE=D:\Python26\include;D:\Python26\include\Stackless > b2 --toolset=msvc-10.0 --build-type=complete stage > b2 --toolset=msvc-10.0 --build-type=complete architecture=x86 > address-model=32 stage > b2 --toolset=msvc-10.0 --with-python --build-type=complete > --debug-configuration architecture=x86 address-model=32 stage > debug-symbols=on debug-store=database python-debugging=on > > > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig From talljimbo at gmail.com Fri Feb 17 23:42:18 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Fri, 17 Feb 2012 17:42:18 -0500 Subject: [C++-sig] Wrapping classes that inherit from classes already wrapped In-Reply-To: <4F3AAE56.2020306@icecube.umd.edu> References: <4F3AAE56.2020306@icecube.umd.edu> Message-ID: <4F3ED7CA.1090403@gmail.com> On 02/14/2012 01:56 PM, Alex Olivas wrote: > Hey All, > I was working on a project a few months ago where I > was wrapping a lib ( using Boost.Python ) that inherited > from classes that were already wrapped ( using SIP ). > Boost.Python however couldn't find the base classes. I > hacked something up ( that I'm not very proud of ) in class.cpp : > > type_handle get_class(type_info id) > { > type_handle result(query_class(id)); > > if (result.get() == 0) > { > PyObject* globals = PyEval_GetGlobals() ; > PyObject* key = object( id.name() ).ptr() ; > PyObject* obj = PyDict_GetItem( globals, key ); > if( obj ){ > return type_handle( obj->ob_type ); > }else{ > object report("extension class wrapper for base class "); > report = report + id.name() + " has not been created yet"; > PyErr_SetObject(PyExc_RuntimeError, report.ptr()); > throw_error_already_set(); > } > } > return result; > } > > > Surely there has to be a better way. Does anyone have a > better solution than this? > SWIG at least doesn't provide any way to go from C++ RTTI information to a Python type object, and that leads me to guess that there probably isn't a way to do it with SIP either (though I really don't know that much about SIP). I agree that your solution above is far from ideal, but I have a hard time imagining anything that's clearly better. > This was also followed by metaclass and layout conflicts, > but that's another issue I think. > It is, but I suspect it's going to be even more difficult to work around in this case, especially layouts. > To give a little background I was trying to wrap a lib that > inherits from Qt and I really would prefer to use the > bindings from PyQt rather than re-wrap ( which also seems > more difficult than I thought it would be ). I think your best bet here is definitely going to be trying to use all SIP or all Boost.Python, at least for classes within one hierarchy. You might be able to write Boost.Python converters for SIP classes so you can pass them to and return them from Boost.Python-wrapped functions, but I think it's going to be very difficult (probably impossible) to have a Boost.Python class inherit from a SIP-wrapped class. Good luck! Jim From johan.rade at gmail.com Sun Feb 19 09:46:05 2012 From: johan.rade at gmail.com (=?ISO-8859-1?Q?Johan_R=E5de?=) Date: Sun, 19 Feb 2012 09:46:05 +0100 Subject: [C++-sig] How to use make_default with keyword expression Message-ID: The following code works fine: using namespace boost::python; class X {}; X* make_X(int n, float f) { return new X; } BOOST_PYTHON_MODULE(Test) { class_("X") .def( "__init__", make_constructor(&make_X) //,(arg("n"), arg("f") = 1.0f) ); } But if I uncomment the commented line, then the code does not compile. How do I fix that? From johan.rade at gmail.com Sun Feb 19 09:49:30 2012 From: johan.rade at gmail.com (=?ISO-8859-1?Q?Johan_R=E5de?=) Date: Sun, 19 Feb 2012 09:49:30 +0100 Subject: [C++-sig] How to use make_default with keyword expression In-Reply-To: References: Message-ID: Stupid me, the header should be "How to use make_constructor with keyword expression". From johan.rade at gmail.com Sun Feb 19 10:39:18 2012 From: johan.rade at gmail.com (=?ISO-8859-1?Q?Johan_R=E5de?=) Date: Sun, 19 Feb 2012 10:39:18 +0100 Subject: [C++-sig] How to use make_default with keyword expression In-Reply-To: References: Message-ID: On 2/19/2012 9:46 AM, Johan R?de wrote: > The following code works fine: > > using namespace boost::python; > > class X {}; > > X* make_X(int n, float f) { return new X; } > > BOOST_PYTHON_MODULE(Test) > { > class_("X") > .def( > "__init__", > make_constructor(&make_X) > //,(arg("n"), arg("f") = 1.0f) > ); > } > > But if I uncomment the commented line, then the code does not compile. > How do I fix that? I figured out how to do it: using namespace boost::python; class X {}; X* make_X(int n, float f) { return new X; } BOOST_PYTHON_MODULE(X) { class_("X") .def( "__init__", make_constructor( &make_X, default_call_policies(), (arg("n"), arg("f") = 1.0f) ) ); } From johan.rade at gmail.com Sun Feb 19 11:18:41 2012 From: johan.rade at gmail.com (=?ISO-8859-1?Q?Johan_R=E5de?=) Date: Sun, 19 Feb 2012 11:18:41 +0100 Subject: [C++-sig] How to use make_default with keyword expression In-Reply-To: References: Message-ID: The documentation of boost::python::make_constructor should be fixed. It shows the synopsis: template object make_constructor(); template object make_constructor(Policies const& policies); This is obviously not correct. From eudoxos at arcig.cz Mon Feb 20 11:13:18 2012 From: eudoxos at arcig.cz (=?utf-8?b?VsOhY2xhdg==?= =?utf-8?b?xaBtaWxhdWVy?=) Date: Mon, 20 Feb 2012 10:13:18 +0000 (UTC) Subject: [C++-sig] Exposing 128-bit aligned datatype Message-ID: I am exposing a struct which is 128-bit aligned to python; it is always constructed dynamically in c++ code, hence there is no trouble that the alignment is correct. I was getting errors on compilation and had to add this bit to my code: namespace boost { namespace align { struct __attribute__((__aligned__(128))) a128 {};} template<> class type_with_alignment<128> { public: typedef align::a128 type; }; }; The wrapper works correctly. For some reason boost/type_traits/type_with_alignment.hpp only defines alignment > 32 only for MSVC or Intel compilers. I am not sure where to ask for fix, can someone forward this to the right place? The aligned type is cl_double16, compiling with gcc. Cheers, Vaclav From talljimbo at gmail.com Mon Feb 20 16:11:15 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Mon, 20 Feb 2012 10:11:15 -0500 Subject: [C++-sig] Exposing 128-bit aligned datatype In-Reply-To: References: Message-ID: <4F426293.3060006@gmail.com> I think this is probably a topic to take to the boost lists (or possibly the boost ticket system, if you're pretty sure it's not intentional). Jim On 02/20/2012 05:13 AM, V?clav?milauer wrote: > I am exposing a struct which is 128-bit aligned to python; it is always > constructed dynamically in c++ code, hence there is no trouble that the > alignment is correct. I was getting errors on compilation and had to add this > bit to my code: > > namespace boost { > namespace align { struct __attribute__((__aligned__(128))) a128 {};} > template<> class type_with_alignment<128> { public: typedef align::a128 type; }; > }; > > The wrapper works correctly. For some reason > boost/type_traits/type_with_alignment.hpp only defines alignment> 32 only for > MSVC or Intel compilers. I am not sure where to ask for fix, can someone forward > this to the right place? > > The aligned type is cl_double16, compiling with gcc. > > Cheers, Vaclav > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig From wright at esrf.fr Mon Feb 20 20:02:31 2012 From: wright at esrf.fr (Jon Wright) Date: Mon, 20 Feb 2012 20:02:31 +0100 Subject: [C++-sig] boost.python: tss_cleanup_implemented link error windows 64 bit Message-ID: <4F4298C7.1070402@esrf.fr> Hello, I have been trying to build pyopencl and pycuda on windows using the subset of boost in those packages on windows 64 bit. The problem is a link error due to missing tss_cleanup_implemented. The codes seem to run without problem if the link error is suppressed: http://thread.gmane.org/gmane.comp.python.cuda/2516 Does anyone know what the tss issue is about and whether it is likely to cause a problem? Any advice would be welcome. Many thanks, Jon From s_sourceforge at nedprod.com Mon Feb 20 22:34:49 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Mon, 20 Feb 2012 21:34:49 -0000 Subject: [C++-sig] Exposing 128-bit aligned datatype In-Reply-To: References: Message-ID: <4F42BC79.22985.D5238C8E@s_sourceforge.nedprod.com> Aligning to 128 bits is surely __attribute__((__aligned__(16)))? You also can inherit from an aligned base class even if that base class contains no items and the alignment passes down just fine. BTW alignment isn't reliable as soon as something leaves static storage e.g. goes through a parameterisation via being given to a function. This is extremely hard to control in C++03 metaprogramming. C++11 makes this much easier as you have rvalues. Also, MSVC is less reliable than GCC with alignment. Use with extreme caution! Generally I have found that the unaligned SSE load ops are sufficiently quick during the occasional misalignment that there's no point being overexact - if misaligned (test via if(addr & 15)), just unaligned load instead of aligned load before executing. Niall On 20 Feb 2012 at 10:13, V?clavSmilauer wrote: > I am exposing a struct which is 128-bit aligned to python; it is always > constructed dynamically in c++ code, hence there is no trouble that the > alignment is correct. I was getting errors on compilation and had to add this > bit to my code: > > namespace boost { > namespace align { struct __attribute__((__aligned__(128))) a128 {};} > template<> class type_with_alignment<128> { public: typedef align::a128 type; }; > }; > > The wrapper works correctly. For some reason > boost/type_traits/type_with_alignment.hpp only defines alignment > 32 only for > MSVC or Intel compilers. I am not sure where to ask for fix, can someone forward > this to the right place? > > The aligned type is cl_double16, compiling with gcc. > > Cheers, Vaclav > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From s_sourceforge at nedprod.com Mon Feb 20 23:42:18 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Mon, 20 Feb 2012 22:42:18 -0000 Subject: [C++-sig] boost.python: tss_cleanup_implemented link error windows 64 bit In-Reply-To: <4F4298C7.1070402@esrf.fr> References: <4F4298C7.1070402@esrf.fr> Message-ID: <4F42CC4A.23025.D561564D@s_sourceforge.nedprod.com> Generally when I see that missing symbol it means mismatched C runtimes. Usually in my experience it's safe to declare that symbol as weak to make the link error go away while handling if it's present correctly. Niall On 20 Feb 2012 at 20:02, Jon Wright wrote: > Hello, > > I have been trying to build pyopencl and pycuda on windows using the > subset of boost in those packages on windows 64 bit. The problem is a > link error due to missing tss_cleanup_implemented. The codes seem to run > without problem if the link error is suppressed: > > http://thread.gmane.org/gmane.comp.python.cuda/2516 > > Does anyone know what the tss issue is about and whether it is likely to > cause a problem? Any advice would be welcome. > > Many thanks, > > Jon > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From adam.preble at gmail.com Tue Feb 21 09:03:56 2012 From: adam.preble at gmail.com (Adam Preble) Date: Tue, 21 Feb 2012 02:03:56 -0600 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info Message-ID: I am using Boost 1.47 I have these class methods: void Remove(const std::type_info &type); void Remove(const boost::python::type_info type); I tried wrapping them both in Boost.Python: void (Entity::*Remove1)(const std::type_info&) = &Foo::Remove; void (Entity::*Remove2)(const boost::python::type_info) = & Foo::Remove; class_< Foo , boost::noncopyable, shared_ptr< Foo > >(" Foo", init<>()) ... .def("Remove", Remove1) .def("Remove", Remove2) ... (I just tried type_info of both namespaces to be comprehensive here...) It'll compile and and I'll get that Remove method fine. Except I can't seem to cram something into it when I try to actually execute the subroutine. Say I have class Bar wrapped in Python too, and I have been using it fine and dandy. How do I get Remove() to accept the type of Bar as an argument? Say: foo = Foo() foo.Remove(Bar)... foo.Remove(type(Bar))... I just can't get any love here: Traceback (most recent call last): File "", line 1, in Boost.Python.ArgumentError: Python argument types in foo.Remove(Foo, type) did not match C++ signature: Remove(class game::Entity {lvalue}, struct boost::python::type_info) Remove(class game::Entity {lvalue}, class type_info) -------------- next part -------------- An HTML attachment was scrubbed... URL: From wright at esrf.fr Tue Feb 21 09:35:38 2012 From: wright at esrf.fr (Jonathan WRIGHT) Date: Tue, 21 Feb 2012 09:35:38 +0100 Subject: [C++-sig] boost.python: tss_cleanup_implemented link error windows 64 bit In-Reply-To: <4F42CC4A.23025.D561564D@s_sourceforge.nedprod.com> References: <4F4298C7.1070402@esrf.fr> <4F42CC4A.23025.D561564D@s_sourceforge.nedprod.com> Message-ID: <4F43575A.2000402@esrf.fr> Many thanks for the quick response. I think I am OK for the runtime as I am using the msvc compiler from the platform SDK. Depends.exe only picks up MSVCP90.dll and MSVCR90.dll and KERNEL32.dll. What is unusual in this package is that boost.python is being linked into the pyd file by adding all the .cpp files in distutils. It seems the link error can be suppressed by defining either: BOOST_THREAD_BUILD_DLL or BOOST_THREAD_BUILD_LIB ...but which one is better, or does it make no difference? We have already defined BOOST_ALL_NO_LIB to disable auto-linking. I guess I am really wondering what on earth is behind the tss issue and whether there is a simple test case to see if we are doing it right. Would something like a little function which creates and destroys threads show a memory leak if the tss is not right? Thanks, Jon On 20/02/2012 23:42, Niall Douglas wrote: > Generally when I see that missing symbol it means mismatched C > runtimes. Usually in my experience it's safe to declare that symbol > as weak to make the link error go away while handling if it's present > correctly. > > Niall > > On 20 Feb 2012 at 20:02, Jon Wright wrote: > >> Hello, >> >> I have been trying to build pyopencl and pycuda on windows using the >> subset of boost in those packages on windows 64 bit. The problem is a >> link error due to missing tss_cleanup_implemented. The codes seem to run >> without problem if the link error is suppressed: >> >> http://thread.gmane.org/gmane.comp.python.cuda/2516 >> >> Does anyone know what the tss issue is about and whether it is likely to >> cause a problem? Any advice would be welcome. >> >> Many thanks, >> >> Jon >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> http://mail.python.org/mailman/listinfo/cplusplus-sig > > From eudoxos at arcig.cz Tue Feb 21 11:39:30 2012 From: eudoxos at arcig.cz (=?utf-8?b?VsOhY2xhdg==?= =?utf-8?b?xaBtaWxhdWVy?=) Date: Tue, 21 Feb 2012 10:39:30 +0000 (UTC) Subject: [C++-sig] Exposing 128-bytes aligned datatype References: <4F42BC79.22985.D5238C8E@s_sourceforge.nedprod.com> Message-ID: > Aligning to 128 bits is surely __attribute__((__aligned__(16)))? Oops, I wrote it wrong in the title, it is 128-bytes aligned, double[16] with natural alignment, i.e. __attribute__((aligned(128))). Thanks for comments, I will try to put it into the bugtracker. From s_sourceforge at nedprod.com Tue Feb 21 14:49:38 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Tue, 21 Feb 2012 13:49:38 -0000 Subject: [C++-sig] boost.python: tss_cleanup_implemented link error windows 64 bit In-Reply-To: <4F43575A.2000402@esrf.fr> References: <4F4298C7.1070402@esrf.fr>, <4F42CC4A.23025.D561564D@s_sourceforge.nedprod.com>, <4F43575A.2000402@esrf.fr> Message-ID: <4F43A0F2.14658.D8A0B4A0@s_sourceforge.nedprod.com> Are you saying that you're statically linking BPL into your PYD? If you are, you shouldn't do that. BPL should live in its own DLL, *especially* when not on Linux/FreeBSD. Niall On 21 Feb 2012 at 9:35, Jonathan WRIGHT wrote: > Many thanks for the quick response. I think I am OK for the runtime as I > am using the msvc compiler from the platform SDK. Depends.exe only picks > up MSVCP90.dll and MSVCR90.dll and KERNEL32.dll. > > What is unusual in this package is that boost.python is being linked > into the pyd file by adding all the .cpp files in distutils. It seems > the link error can be suppressed by defining either: > > BOOST_THREAD_BUILD_DLL or BOOST_THREAD_BUILD_LIB > > ...but which one is better, or does it make no difference? > We have already defined BOOST_ALL_NO_LIB to disable auto-linking. > > I guess I am really wondering what on earth is behind the tss issue and > whether there is a simple test case to see if we are doing it right. > Would something like a little function which creates and destroys > threads show a memory leak if the tss is not right? > > Thanks, > > Jon > > > On 20/02/2012 23:42, Niall Douglas wrote: > > Generally when I see that missing symbol it means mismatched C > > runtimes. Usually in my experience it's safe to declare that symbol > > as weak to make the link error go away while handling if it's present > > correctly. > > > > Niall > > > > On 20 Feb 2012 at 20:02, Jon Wright wrote: > > > >> Hello, > >> > >> I have been trying to build pyopencl and pycuda on windows using the > >> subset of boost in those packages on windows 64 bit. The problem is a > >> link error due to missing tss_cleanup_implemented. The codes seem to run > >> without problem if the link error is suppressed: > >> > >> http://thread.gmane.org/gmane.comp.python.cuda/2516 > >> > >> Does anyone know what the tss issue is about and whether it is likely to > >> cause a problem? Any advice would be welcome. > >> > >> Many thanks, > >> > >> Jon > >> _______________________________________________ > >> Cplusplus-sig mailing list > >> Cplusplus-sig at python.org > >> http://mail.python.org/mailman/listinfo/cplusplus-sig > > > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From s_sourceforge at nedprod.com Tue Feb 21 14:49:38 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Tue, 21 Feb 2012 13:49:38 -0000 Subject: [C++-sig] Exposing 128-bytes aligned datatype In-Reply-To: References: , Message-ID: <4F43A0F2.27816.D8A0B52C@s_sourceforge.nedprod.com> On 21 Feb 2012 at 10:39, V?clavSmilauer wrote: > > Aligning to 128 bits is surely __attribute__((__aligned__(16)))? > > Oops, I wrote it wrong in the title, it is 128-bytes aligned, double[16] with > natural alignment, i.e. __attribute__((aligned(128))). > > Thanks for comments, I will try to put it into the bugtracker. 128 *byte* alignment? Wow. No, no compiler will support that legally as it would crap all over your stack frame. I vaguely remember that GCC caps alignment to 40 bytes due to some supported architecture's stack frame being incapable of going higher. If you're aligning to 128 bytes then you almost certainly need to revisit how your code is designed and what algorithms are being used. I can think of two occasions ever in my life where I have seen an alignment larger than 32 bytes being a reasonably good idea, and one of those was in ptmalloc2 (which masks off the bottom megabyte of bits as a very fast way of finding the segment for a pointer). And even in ptmalloc2 I am unconvinced that approach was wise due to many reasons unimportant to mention here. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From wright at esrf.fr Tue Feb 21 16:30:26 2012 From: wright at esrf.fr (Jonathan WRIGHT) Date: Tue, 21 Feb 2012 16:30:26 +0100 Subject: [C++-sig] boost.python: tss_cleanup_implemented link error windows 64 bit In-Reply-To: <4F43A0F2.14658.D8A0B4A0@s_sourceforge.nedprod.com> References: <4F4298C7.1070402@esrf.fr>, <4F42CC4A.23025.D561564D@s_sourceforge.nedprod.com>, <4F43575A.2000402@esrf.fr> <4F43A0F2.14658.D8A0B4A0@s_sourceforge.nedprod.com> Message-ID: <4F43B892.4020109@esrf.fr> Boost python is being compiled in .obj files and then the whole lot is linked into a .pyd. Do you have some unit tests we could use to check our builds for problems? I've not understood how the dll linking requirement works. Where should the dll live and how do you then cope with other people's boost based modules, which might link to a different boost dll version? Many thanks for your help! Jon On 21/02/2012 14:49, Niall Douglas wrote: > Are you saying that you're statically linking BPL into your PYD? If > you are, you shouldn't do that. BPL should live in its own DLL, > *especially* when not on Linux/FreeBSD. > > Niall > > > On 21 Feb 2012 at 9:35, Jonathan WRIGHT wrote: > >> Many thanks for the quick response. I think I am OK for the runtime as I >> am using the msvc compiler from the platform SDK. Depends.exe only picks >> up MSVCP90.dll and MSVCR90.dll and KERNEL32.dll. >> >> What is unusual in this package is that boost.python is being linked >> into the pyd file by adding all the .cpp files in distutils. It seems >> the link error can be suppressed by defining either: >> >> BOOST_THREAD_BUILD_DLL or BOOST_THREAD_BUILD_LIB >> >> ...but which one is better, or does it make no difference? >> We have already defined BOOST_ALL_NO_LIB to disable auto-linking. >> >> I guess I am really wondering what on earth is behind the tss issue and >> whether there is a simple test case to see if we are doing it right. >> Would something like a little function which creates and destroys >> threads show a memory leak if the tss is not right? >> >> Thanks, >> >> Jon >> >> >> On 20/02/2012 23:42, Niall Douglas wrote: >>> Generally when I see that missing symbol it means mismatched C >>> runtimes. Usually in my experience it's safe to declare that symbol >>> as weak to make the link error go away while handling if it's present >>> correctly. >>> >>> Niall >>> >>> On 20 Feb 2012 at 20:02, Jon Wright wrote: >>> >>>> Hello, >>>> >>>> I have been trying to build pyopencl and pycuda on windows using the >>>> subset of boost in those packages on windows 64 bit. The problem is a >>>> link error due to missing tss_cleanup_implemented. The codes seem to run >>>> without problem if the link error is suppressed: >>>> >>>> http://thread.gmane.org/gmane.comp.python.cuda/2516 >>>> >>>> Does anyone know what the tss issue is about and whether it is likely to >>>> cause a problem? Any advice would be welcome. >>>> >>>> Many thanks, >>>> >>>> Jon >>>> _______________________________________________ >>>> Cplusplus-sig mailing list >>>> Cplusplus-sig at python.org >>>> http://mail.python.org/mailman/listinfo/cplusplus-sig >>> >>> >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> http://mail.python.org/mailman/listinfo/cplusplus-sig > > From eudoxos at arcig.cz Tue Feb 21 17:46:58 2012 From: eudoxos at arcig.cz (=?utf-8?b?VsOhY2xhdg==?= =?utf-8?b?xaBtaWxhdWVy?=) Date: Tue, 21 Feb 2012 16:46:58 +0000 (UTC) Subject: [C++-sig] Exposing 128-bytes aligned datatype References: , <4F43A0F2.27816.D8A0B52C@s_sourceforge.nedprod.com> Message-ID: > 128 *byte* alignment? Wow. No, no compiler will support that legally > as it would crap all over your stack frame. I vaguely remember that > GCC caps alignment to 40 bytes due to some supported architecture's > stack frame being incapable of going higher. > If you're aligning to 128 bytes then you almost certainly need to > revisit how your code is designed and what algorithms are being used. I did not design the OpenCL standard (see http://www.khronos.org/registry/cl/api/1.2/cl_platform.h at the end, there is cl_double16). Just to assure you, those types are never ever passed on stack. That header works just fine with gcc. From s_sourceforge at nedprod.com Wed Feb 22 00:49:24 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Tue, 21 Feb 2012 23:49:24 -0000 Subject: [C++-sig] boost.python: tss_cleanup_implemented link error windows 64 bit In-Reply-To: <4F43B892.4020109@esrf.fr> References: <4F4298C7.1070402@esrf.fr>, <4F43A0F2.14658.D8A0B4A0@s_sourceforge.nedprod.com>, <4F43B892.4020109@esrf.fr> Message-ID: <4F442D84.32225.DAC5CF7C@s_sourceforge.nedprod.com> On 21 Feb 2012 at 16:30, Jonathan WRIGHT wrote: > Boost python is being compiled in .obj files and then the whole lot is > linked into a .pyd. > > Do you have some unit tests we could use to check our builds for > problems? I've not understood how the dll linking requirement works. > Where should the dll live and how do you then cope with other people's > boost based modules, which might link to a different boost dll version? There is plenty of information about the pros and cons and what to do in the archives of this mailing list which is google searchable. In short, static linking means BPL based python extension modules must not interact outside of python or it'll go tits up due to their type registries not knowing about one another. That means you can't have extension A and extension B working together directly, only through python. If it's working for you right now I wouldn't worry. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From s_sourceforge at nedprod.com Wed Feb 22 00:49:24 2012 From: s_sourceforge at nedprod.com (Niall Douglas) Date: Tue, 21 Feb 2012 23:49:24 -0000 Subject: [C++-sig] Exposing 128-bytes aligned datatype In-Reply-To: References: , Message-ID: <4F442D84.21777.DAC5CEE0@s_sourceforge.nedprod.com> On 21 Feb 2012 at 16:46, V?clavSmilauer wrote: > > 128 *byte* alignment? Wow. No, no compiler will support that legally > > as it would crap all over your stack frame. I vaguely remember that > > GCC caps alignment to 40 bytes due to some supported architecture's > > stack frame being incapable of going higher. > > If you're aligning to 128 bytes then you almost certainly need to > > revisit how your code is designed and what algorithms are being used. > > I did not design the OpenCL standard (see > http://www.khronos.org/registry/cl/api/1.2/cl_platform.h at the end, there is > cl_double16). Just to assure you, those types are never ever passed on stack. > That header works just fine with gcc. I agree that GCC/MSVC needs two alignment specifiers, one for static data alignment and another for everything else. This would be a good candidate for inclusion into the C programming language actually. I may propose it to WG14. Niall -- Technology & Consulting Services - ned Productions Limited. http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 472909. From adam.preble at gmail.com Thu Feb 23 08:31:56 2012 From: adam.preble at gmail.com (Adam Preble) Date: Thu, 23 Feb 2012 01:31:56 -0600 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info In-Reply-To: References: Message-ID: I think I need to ask a better question here, so I'm coming back here with an adjusted situation. I've found I can't get Python to bind up to prototypes for stuff taking std::type_info or boost::type_info. I suppose I should be surprised if it actually did. I'm thinking--what if I have a Python implementation of an interface I defined in C++? It would be really odd if that would even be compatible with type_info in any way. I am thinking type_info is not the way to go, but I see some code in Boost dealing with PyType. I figure what I need instead is a method that types something else than type_info, but what I need is a mystery. For giggles, I just tried to take a PyObject* and pass in whatever. if I give it from Python type(Foo) then I think I get a PyType. I see some stuff in Boost source code about working with these. I'm wondering, is there any helpers for this? Or any idea at all what to do? I suppose the overall situation is this: imagine I have a container of pointers referencing some interface. The container contains potentially both C++ and Python implementations. All implementations are exposed in Python. Is there a way I could pass in some kind of type information and be able to positively identify if something in that container matches the type? On Tue, Feb 21, 2012 at 2:03 AM, Adam Preble wrote: > It'll compile and and I'll get that Remove method fine. Except I can't > seem to cram something into it when I try to actually execute the > subroutine. Say I have class Bar wrapped in Python too, and I have been > using it fine and dandy. How do I get Remove() to accept the type of Bar > as an argument? > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Thu Feb 23 17:03:18 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Thu, 23 Feb 2012 11:03:18 -0500 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info In-Reply-To: References: Message-ID: <4F466346.3090505@gmail.com> On 02/23/2012 02:31 AM, Adam Preble wrote: > I think I need to ask a better question here, so I'm coming back here with > an adjusted situation. I've found I can't get Python to bind up to > prototypes for stuff taking std::type_info or boost::type_info. I suppose > I should be surprised if it actually did. I'm thinking--what if I have a > Python implementation of an interface I defined in C++? It would be really > odd if that would even be compatible with type_info in any way. I am > thinking type_info is not the way to go, but I see some code in Boost > dealing with PyType. > Glad you sent this; I was about to reply to your last with something that probably wouldn't have been very helpful, for the reasons you've already noted yourself. > I figure what I need instead is a method that types something else than > type_info, but what I need is a mystery. For giggles, I just tried to take > a PyObject* and pass in whatever. if I give it from Python type(Foo) then > I think I get a PyType. I see some stuff in Boost source code about > working with these. I'm wondering, is there any helpers for this? Or any > idea at all what to do? > You can probably find out a lot from just looking at the Python C API reference documentation for type objects. You'll need to use that directly for a lot of your interaction with them, because there's no Boost.Python wrapper (like there is for, say, tuple or str). The main place you'll see them in Boost.Python is that this is actually what a boost::python::class_ object holds under the hood; you can assign that to a boost::python::object to get the type object without having to carry around all the template parameters. > I suppose the overall situation is this: imagine I have a container of > pointers referencing some interface. The container contains potentially > both C++ and Python implementations. All implementations are exposed in > Python. Is there a way I could pass in some kind of type information and > be able to positively identify if something in that container matches the > type? > Yes, as long as: - You can do the checking with a Python function that operates on type objects, like isinstance and issubclass (or their C API equivalents, PyObject_IsInstance and PyObject_IsSubclass). - You are content with looking up a PyTypeObject* given a C++ type_info, and not the other way around; you can use boost::python::converter::registry::lookup to find the PyTypeObject for a Boost.Python-wrapped class. That's deep in the bowels of Boost.Python's internals - see converter/registry.hpp and converter/registration.hpp to learn more - but it's quite usable. Jim From Holger.Joukl at LBBW.de Thu Feb 23 17:50:49 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Thu, 23 Feb 2012 17:50:49 +0100 Subject: [C++-sig] test if object instance is instance of extension class in C++ Message-ID: Hi, what's the recommended way to check if an object instance is an instance of my extension class in C++ code? I'm currently doing a very ugly #define isMyExtensionClass(pyobj) (strcmp(pyobj->ob_type->tp_name, "MyExtensionClass") == 0) on PyObj* pyobj. I bet there is a better way but just can't seem to find it. Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From Holger.Joukl at LBBW.de Thu Feb 23 18:04:44 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Thu, 23 Feb 2012 18:04:44 +0100 Subject: [C++-sig] how to override __setattr__ of an extension class Message-ID: Hi, I'm trying to instrument a wrapped class with custom __setattr__ mechanics: .def("__setattr__", &MyExtensionClass::setattr) How can I call the base class __setattr__ from within setattr and not run into infinite recursion? I.e. the analogon in C++ to >> class Foo(object): ... def __setattr__(self, attr, value): ... print "custom __setattr__" ... object.__setattr__(self, attr, value) ... >>> foo = Foo() >>> foo.x = 23 custom __setattr__ >>> foo.x 23 >>> To actually write to my instance dict I currently do PyObject* __dict__ = PyObject_GetAttrString(m_self, const_cast ("__dict__")); PyDict_SetItemString(__dict__, name, value.ptr()); inside MyExtensionClass::setattr (where m_self is PyObj*) Is there a better way? On a sidenote, I initially thought I could wrap the __dict__ in bp::dict() to use its nice Python-like API for item assignment but this seems to always make a copy of the __dict__. I.e something like bp::dict bp_dict(__dict__); bp_dict[name] = value; won't work. Regards Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From stefan at seefeld.name Thu Feb 23 18:12:53 2012 From: stefan at seefeld.name (Stefan Seefeld) Date: Thu, 23 Feb 2012 12:12:53 -0500 Subject: [C++-sig] test if object instance is instance of extension class in C++ In-Reply-To: References: Message-ID: <4F467395.4050702@seefeld.name> On 2012-02-23 11:50, Holger Joukl wrote: > Hi, > > what's the recommended way to check if an object instance is an instance of > my extension class > in C++ code? > > I'm currently doing a very ugly > > #define isMyExtensionClass(pyobj) (strcmp(pyobj->ob_type->tp_name, > "MyExtensionClass") == 0) > > on PyObj* pyobj. > > I bet there is a better way but just can't seem to find it. bpl::extract e(pyobj); if (e.check()) ... Note that you may want to use a reference type as template argument to the boost::python::extract type, if you want to retrieve the object by reference instead of by value. See http://www.boost.org/doc/libs/1_40_0/libs/python/doc/v2/extract.html for details. HTH, Stefan -- ...ich hab' noch einen Koffer in Berlin... From adam.preble at gmail.com Fri Feb 24 08:31:49 2012 From: adam.preble at gmail.com (Adam Preble) Date: Fri, 24 Feb 2012 01:31:49 -0600 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info In-Reply-To: <4F466346.3090505@gmail.com> References: <4F466346.3090505@gmail.com> Message-ID: Jim, I'll top post since I think this might get messy nice and fast otherwise. I found I couldn't get to a method wrapped to type a PyTypeObject* and only PyObject* would do. Despite all this, I managed to twist your advice around just enough to birth this abomination: bool ContainsImplementables::Has(PyObject* noIdea) { for(std::list >::iterator i = implementables.begin(); i != implementables.end(); ++i) { Implementable* imp = (*i).get(); boost::python::converter::registration registration = boost::python::converter::registry::lookup(boost::python::type_info(typeid(*imp))); PyTypeObject* internal_type = const_cast(registration.to_python_target_type()); // const_cast heresy if(internal_type != NULL) { if(PyObject_IsInstance((PyObject*) internal_type, noIdea) == 1) { return true; } } } return false; } Well, it doesn't crash! I really don't know if I'm getting all the right things in order, but I know in the debugger the stuff doesn't look like complete garbage, and internal_type isn't NULL. The PyObject_IsInstance() is never working when I pass in a class to Has() and it's always claiming true if I pass type(class) to the Has() method there, regardless if the particular class in question is actually contained inside there. I tried to step into that particular code and it was pretty rough on the eyes; I couldn't figure out anything. So to be more explicit about what happens: cont = ContainsImplementables() cont.Add(CImplementation()) cont.Add(PythonImplementation()) cont.Has(CImplementation) # False cont.Has(type(CImplementation)) # True cont.Has(PythonImplementation) # False cont.Has(type( Python Implementation)) # True cont.Has(AnotherPythonImplementation) # False cont.Has(type(AnotherPythonImplementation)) # True Am I anywhere near close on this? On Thu, Feb 23, 2012 at 10:03 AM, Jim Bosch wrote: > > > You can probably find out a lot from just looking at the Python C API > reference documentation for type objects. You'll need to use that directly > for a lot of your interaction with them, because there's no Boost.Python > wrapper (like there is for, say, tuple or str). > > The main place you'll see them in Boost.Python is that this is actually > what a boost::python::class_ object holds under the hood; you can assign > that to a boost::python::object to get the type object without having to > carry around all the template parameters. > > > I suppose the overall situation is this: imagine I have a container of >> pointers referencing some interface. The container contains potentially >> both C++ and Python implementations. All implementations are exposed in >> Python. Is there a way I could pass in some kind of type information and >> be able to positively identify if something in that container matches the >> type? >> >> > Yes, as long as: > > - You can do the checking with a Python function that operates on type > objects, like isinstance and issubclass (or their C API equivalents, > PyObject_IsInstance and PyObject_IsSubclass). > > - You are content with looking up a PyTypeObject* given a C++ type_info, > and not the other way around; you can use boost::python::converter::**registry::lookup > to find the PyTypeObject for a Boost.Python-wrapped class. That's deep in > the bowels of Boost.Python's internals - see converter/registry.hpp and > converter/registration.hpp to learn more - but it's quite usable. > > > > Jim > > ______________________________**_________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/**mailman/listinfo/cplusplus-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Holger.Joukl at LBBW.de Fri Feb 24 11:44:40 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Fri, 24 Feb 2012 11:44:40 +0100 Subject: [C++-sig] test if object instance is instance of extension class in C++ In-Reply-To: <4F467395.4050702@seefeld.name> References: <4F467395.4050702@seefeld.name> Message-ID: Stefan, many thanks, that of course did the trick. I'm not seeing the wood for the trees any more, I fear. > > what's the recommended way to check if an object instance is an instance of > > my extension class > > in C++ code? > > > > I'm currently doing a very ugly > > > > #define isMyExtensionClass(pyobj) (strcmp(pyobj->ob_type->tp_name, > > "MyExtensionClass") == 0) > > > > on PyObj* pyobj. > > > > I bet there is a better way but just can't seem to find it. > > bpl::extract e(pyobj); > if (e.check()) ... > > Note that you may want to use a reference type as template argument to > the boost::python::extract type, if you want to retrieve the object by > reference instead of by value. > > See http://www.boost.org/doc/libs/1_40_0/libs/python/doc/v2/extract.html > for details. Just for the record, this is what I do now: inline bool isMyExtensionClass(PyObject* &pyobj) { bp::extract extractor(pyobj); return extractor.check(); } inline bool isMyExtensionClass(bp::object &obj) { bp::extract extractor(obj); return extractor.check(); } Thanks again & have a nice weekend Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From stefan at seefeld.name Fri Feb 24 14:36:27 2012 From: stefan at seefeld.name (Stefan Seefeld) Date: Fri, 24 Feb 2012 08:36:27 -0500 Subject: [C++-sig] test if object instance is instance of extension class in C++ In-Reply-To: References: <4F467395.4050702@seefeld.name> Message-ID: <4F47925B.7020805@seefeld.name> On 02/24/2012 05:44 AM, Holger Joukl wrote: > Just for the record, this is what I do now: > > inline > bool isMyExtensionClass(PyObject* &pyobj) { > bp::extract extractor(pyobj); > return extractor.check(); > } I'm not sure why you use raw PyObject pointers in the first place. That should all be hidden behind bpl::object instances (which will then take care of the ref counting business for you). > inline > bool isMyExtensionClass(bp::object &obj) { > bp::extract extractor(obj); > return extractor.check(); > } bp::object itself has reference (smart pointer) semantics, so there is no need to pass objects by reference. Stefan -- ...ich hab' noch einen Koffer in Berlin... From Holger.Joukl at LBBW.de Fri Feb 24 15:33:49 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Fri, 24 Feb 2012 15:33:49 +0100 Subject: [C++-sig] test if object instance is instance of extension class in C++ In-Reply-To: <4F47925B.7020805@seefeld.name> References: <4F467395.4050702@seefeld.name> <4F47925B.7020805@seefeld.name> Message-ID: Hi, > On 02/24/2012 05:44 AM, Holger Joukl wrote: > > Just for the record, this is what I do now: > > > > inline > > bool isMyExtensionClass(PyObject* &pyobj) { > > bp::extract extractor(pyobj); > > return extractor.check(); > > } > > I'm not sure why you use raw PyObject pointers in the first place. That > should all be hidden behind bpl::object instances (which will then take > care of the ref counting business for you). We're porting a legacy boost.python v1-wrapped library to current boost.python. Some parts of its internal methods operate on raw PyObject pointers. While we're probably going to change that for now I all I want is a running version and change as little as possible. > > inline > > bool isMyExtensionClass(bp::object &obj) { > > bp::extract extractor(obj); > > return extractor.check(); > > } > > bp::object itself has reference (smart pointer) semantics, so there is > no need to pass objects by reference. I see. Just out of curiosity: If I pass by reference I do save a copy constructor call, don't I? But I circumvent the bp::object refcount safety, though(?). But in a type check like above this shouldn't be dangerous - right? Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From stefan at seefeld.name Fri Feb 24 15:49:48 2012 From: stefan at seefeld.name (Stefan Seefeld) Date: Fri, 24 Feb 2012 09:49:48 -0500 Subject: [C++-sig] test if object instance is instance of extension class in C++ In-Reply-To: References: <4F467395.4050702@seefeld.name> <4F47925B.7020805@seefeld.name> Message-ID: <4F47A38C.6090103@seefeld.name> On 02/24/2012 09:33 AM, Holger Joukl wrote: > bp::object itself has reference (smart pointer) semantics, so there is > no need to pass objects by reference. > I see. Just out of curiosity: > If I pass by reference I do save a copy constructor call, don't I? In principle, yes. I'm not sure how much of this the compiler would be able to optimize away, though. > But I circumvent the bp::object refcount safety, though(?). Well, yes, but you don't need that as C++ language semantics ensure the object lifetime. > But in a type check like above this shouldn't be dangerous - right? There is definitely no danger in doing this. It may just add a little bit of overhead (since in certain cases passing by value is cheaper than passing by reference, depending on the involved types and optimization level. Stefan -- ...ich hab' noch einen Koffer in Berlin... From Holger.Joukl at LBBW.de Fri Feb 24 17:45:42 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Fri, 24 Feb 2012 17:45:42 +0100 Subject: [C++-sig] test if object instance is instance of extension class in C++ In-Reply-To: <4F47A38C.6090103@seefeld.name> References: <4F467395.4050702@seefeld.name> <4F47925B.7020805@seefeld.name> <4F47A38C.6090103@seefeld.name> Message-ID: > > bp::object itself has reference (smart pointer) semantics, so there is > > no need to pass objects by reference. > > I see. Just out of curiosity: > > If I pass by reference I do save a copy constructor call, don't I? > > In principle, yes. I'm not sure how much of this the compiler would be > able to optimize away, though. > > > But I circumvent the bp::object refcount safety, though(?). > > Well, yes, but you don't need that as C++ language semantics ensure the > object lifetime. > > > But in a type check like above this shouldn't be dangerous - right? > > There is definitely no danger in doing this. It may just add a little > bit of overhead (since in certain cases passing by value is cheaper than > passing by reference, depending on the involved types and optimization > level. Ok, this sounds like its best to do inline bool isMyExtensionClass(PyObject* pyobj) { bp::extract extractor(pyobj); return extractor.check(); } inline bool isMyExtensionClass(bp::object obj) { bp::extract extractor(obj); return extractor.check(); } Many thanks again Stefan, Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From talljimbo at gmail.com Sat Feb 25 04:19:59 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Fri, 24 Feb 2012 22:19:59 -0500 Subject: [C++-sig] how to override __setattr__ of an extension class In-Reply-To: References: Message-ID: <4F48535F.6070803@gmail.com> On 02/23/2012 12:04 PM, Holger Joukl wrote: > > Hi, > > I'm trying to instrument a wrapped class with custom __setattr__ > mechanics: > > .def("__setattr__",&MyExtensionClass::setattr) > > How can I call the base class __setattr__ from within setattr and not run > into infinite recursion? > > I.e. the analogon in C++ to > >>> class Foo(object): > ... def __setattr__(self, attr, value): > ... print "custom __setattr__" > ... object.__setattr__(self, attr, value) > ... >>>> foo = Foo() >>>> foo.x = 23 > custom __setattr__ >>>> foo.x > 23 >>>> > > To actually write to my instance dict I currently do > > PyObject* __dict__ = PyObject_GetAttrString(m_self, const_cast > ("__dict__")); > PyDict_SetItemString(__dict__, name, value.ptr()); > > inside MyExtensionClass::setattr (where m_self is PyObj*) > > Is there a better way? > Everything I can think of (e.g. extract the base class from __bases__[0] and call its setattr) is functionally equivalent and not any prettier. > On a sidenote, I initially thought I could wrap the __dict__ in bp::dict() > to > use its nice Python-like API for item assignment but this seems to always > make a copy of the __dict__. > I.e something like > > bp::dict bp_dict(__dict__); > bp_dict[name] = value; > > won't work. > The trick is to do: bp::dict bp_dict = bp::extract(__dict__); bp_dict[name] = value; Just calling the bp::dict invokes Python's dict constructor, which does a copy. Jim From talljimbo at gmail.com Sat Feb 25 04:56:57 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Fri, 24 Feb 2012 22:56:57 -0500 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info In-Reply-To: References: <4F466346.3090505@gmail.com> Message-ID: <4F485C09.6020706@gmail.com> On 02/24/2012 02:31 AM, Adam Preble wrote: > Jim, > > I'll top post since I think this might get messy nice and fast > otherwise. I found I couldn't get to a method wrapped to type a > PyTypeObject* and only PyObject* would do. Yeah, that's not surprising. You can also have it take a boost::python::object; that's essentially a smart pointer for PyObject*. > if(PyObject_IsInstance((PyObject*) internal_type, noIdea) == 1) I think this is the only line that needs to change; you just need to use PyObject_IsSubClass instead of PyObject_IsInstance. Right now you're asking whether "internal_type" is an instance of "noIdea", so the answer is only True when "noIdea" is "type". I think you want to ask whether "internal_type" is a subclass of "noIdea". And that's just: if (PyObject_IsInstance((PyObject*)internal_type, noIdea) == 1) Jim From adam.preble at gmail.com Sat Feb 25 15:25:13 2012 From: adam.preble at gmail.com (Adam Preble) Date: Sat, 25 Feb 2012 08:25:13 -0600 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info In-Reply-To: <4F485C09.6020706@gmail.com> References: <4F466346.3090505@gmail.com> <4F485C09.6020706@gmail.com> Message-ID: It looks like PyObject_IsSubClass is working (!), but I wanted to follow up on the boost::python::object stuff: On Fri, Feb 24, 2012 at 9:56 PM, Jim Bosch wrote: > On 02/24/2012 02:31 AM, Adam Preble wrote: > >> Jim, >> >> I'll top post since I think this might get messy nice and fast >> otherwise. I found I couldn't get to a method wrapped to type a >> PyTypeObject* and only PyObject* would do. >> > > Yeah, that's not surprising. You can also have it take a > boost::python::object; that's essentially a smart pointer for PyObject*. > I couldn't get that to work. I can't pass it directly into PyObject_IsSubclass, which isn't surprising. However, I couldn't do a get or find anything else to extract the PyObject* out of it. Is there a trick to this? I would wonder--is there a particular reason I should use that instead? In my case, I'm not trying to sit on the pointer outside of the class to Has(). The list I'm using has shared_ptr's of the underlying type already. So I assume in my particular case I wouldn't have to worry about it. > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From talljimbo at gmail.com Sat Feb 25 18:08:23 2012 From: talljimbo at gmail.com (Jim Bosch) Date: Sat, 25 Feb 2012 12:08:23 -0500 Subject: [C++-sig] Wrapping a function that takes class type, such as type_info In-Reply-To: References: <4F466346.3090505@gmail.com> <4F485C09.6020706@gmail.com> Message-ID: <4F491587.2000505@gmail.com> On 02/25/2012 09:25 AM, Adam Preble wrote: > It looks like PyObject_IsSubClass is working (!), but I wanted to follow up > on the boost::python::object stuff: > > On Fri, Feb 24, 2012 at 9:56 PM, Jim Bosch wrote: > >> On 02/24/2012 02:31 AM, Adam Preble wrote: >> >>> Jim, >>> >>> I'll top post since I think this might get messy nice and fast >>> otherwise. I found I couldn't get to a method wrapped to type a >>> PyTypeObject* and only PyObject* would do. >>> >> >> Yeah, that's not surprising. You can also have it take a >> boost::python::object; that's essentially a smart pointer for PyObject*. >> > > I couldn't get that to work. I can't pass it directly into > PyObject_IsSubclass, which isn't surprising. However, I couldn't do a get > or find anything else to extract the PyObject* out of it. Is there a trick > to this? I would wonder--is there a particular reason I should use that > instead? In my case, I'm not trying to sit on the pointer outside of the > class to Has(). The list I'm using has shared_ptr's of the underlying type > already. So I assume in my particular case I wouldn't have to worry about > it. > It's - probably unfortunately - just a totally non-standard accessor: ".ptr()". But you're basically right - there's really no need to worry about managing the pointer lifetime in this case, so PyObject * is just fine. Jim From rwgrosse-kunstleve at lbl.gov Mon Feb 27 03:21:18 2012 From: rwgrosse-kunstleve at lbl.gov (Ralf Grosse-Kunstleve) Date: Sun, 26 Feb 2012 18:21:18 -0800 Subject: [C++-sig] How to use make_default with keyword expression In-Reply-To: References: Message-ID: Thanks for pointing this out. Fixed with boost svn trunk revision 77124. Ralf On Sun, Feb 19, 2012 at 2:18 AM, Johan R?de wrote: > The documentation of boost::python::make_**constructor should be fixed. > It shows the synopsis: > > template > object make_constructor(); > > template > object make_constructor(Policies const& policies); > > This is obviously not correct. > > > ______________________________**_________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/**mailman/listinfo/cplusplus-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From Holger.Joukl at LBBW.de Tue Feb 28 14:17:47 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Tue, 28 Feb 2012 14:17:47 +0100 Subject: [C++-sig] how to override __setattr__ of an extension class In-Reply-To: <4F48535F.6070803@gmail.com> References: <4F48535F.6070803@gmail.com> Message-ID: > > To actually write to my instance dict I currently do > > > > PyObject* __dict__ = PyObject_GetAttrString(m_self, const_cast > > ("__dict__")); > > PyDict_SetItemString(__dict__, name, value.ptr()); > > > > inside MyExtensionClass::setattr (where m_self is PyObj*) > > > > Is there a better way? > > > > Everything I can think of (e.g. extract the base class from __bases__[0] > and call its setattr) is functionally equivalent and not any prettier. Good to know. I'll stick to this, then. > The trick is to do: > > bp::dict bp_dict = bp::extract(__dict__); > bp_dict[name] = value; > > Just calling the bp::dict invokes Python's dict constructor, which does > a copy. ...and now that you've told me what to do I've found that it's actually documented here: http://www.boost.org/doc/libs/1_48_0/libs/python/doc/tutorial/doc/html/python/object.html#python.extracting_c___objects: "[...] The astute reader might have noticed that the extract facility in fact solves the mutable copying problem: dict d = extract(x.attr("__dict__")); d["whatever"] = 3; // modifies x.__dict__ ! [...]" and there: http://www.boost.org/doc/libs/1_48_0/libs/python/doc/v2/extract.html: "[...] Because invoking a mutable python type with an argument of the same type (e.g. list([1,2]) typically makes a copy of the argument object, this may be the only way to access the ObjectWrapper's interface on the original object.[...]" Maybe that's only me but I don't think the info that gets you going here is very obvious in the docs, more like a casual side note. So for now I've added some info on this here: http://wiki.python.org/moin/boost.python/extract Many thanks Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From wmamrak at gmail.com Tue Feb 28 15:35:26 2012 From: wmamrak at gmail.com (Wojciech Mamrak) Date: Tue, 28 Feb 2012 15:35:26 +0100 Subject: [C++-sig] how to override __setattr__ of an extension class In-Reply-To: References: <4F48535F.6070803@gmail.com> Message-ID: Here is a part of my message posted here few months ago, which has been left without any response (no offence - maybe it disappeared somewhere): The docs regarding the mutable copying problem are very misleading. Following Python [1] and Boost [2] scripts do not produce the same results: [1] b = a = [] c = list(a) a.append("s1") print a.count("s1"), b.count("s1"), c.count("s1") #prints 1 1 0 [2] list a, b, c; b = a; c = list(a); a.append("s1"); list d = extract(a); printf("%i %i %i %i\n", a.count("s1"), b.count("s1"), c.count("s1"), d.count("s1")); //prints 1 1 1 1 In [2] expected was 1 1 0 1 according to Pythonic behaviour (it states in docs that such behaviour is mimicked). This is in conflict with example provided in the tutorial. regards 2012/2/28 Holger Joukl : >> > To actually write to my instance dict I currently do >> > >> > ? ? ?PyObject* __dict__ = PyObject_GetAttrString(m_self, > const_cast >> > ("__dict__")); >> > ? ? ?PyDict_SetItemString(__dict__, name, value.ptr()); >> > >> > inside MyExtensionClass::setattr (where m_self is PyObj*) >> > >> > Is there a better way? >> > >> >> Everything I can think of (e.g. extract the base class from __bases__[0] >> and call its setattr) is functionally equivalent and not any prettier. > > Good to know. I'll stick to this, then. > >> The trick is to do: >> >> bp::dict bp_dict = bp::extract(__dict__); >> bp_dict[name] = value; >> >> Just calling the bp::dict invokes Python's dict constructor, which does >> a copy. > > ...and now that you've told me what to do I've found that it's actually > documented > here: > > http://www.boost.org/doc/libs/1_48_0/libs/python/doc/tutorial/doc/html/python/object.html#python.extracting_c___objects: > > "[...] > The astute reader might have noticed that the extract facility in fact > solves the mutable copying problem: > > > dict d = extract(x.attr("__dict__")); > d["whatever"] = 3; ? ? ? ? ?// modifies x.__dict__ ! > [...]" > > > and there: > > > > http://www.boost.org/doc/libs/1_48_0/libs/python/doc/v2/extract.html: > "[...] Because invoking a mutable python type with an argument of the same > type (e.g. list([1,2]) typically makes a copy of the argument object, this > may be the only way to access the ObjectWrapper's interface on the original > object.[...]" > > Maybe that's only me but I don't think the info that gets you going here is > very obvious in the docs, more like > a casual side note. So for now I've added some info on this here: > http://wiki.python.org/moin/boost.python/extract > > Many thanks > Holger > > Landesbank Baden-Wuerttemberg > Anstalt des oeffentlichen Rechts > Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz > HRA 12704 > Amtsgericht Stuttgart > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig From Holger.Joukl at LBBW.de Tue Feb 28 17:52:46 2012 From: Holger.Joukl at LBBW.de (Holger Joukl) Date: Tue, 28 Feb 2012 17:52:46 +0100 Subject: [C++-sig] how to override __setattr__ of an extension class In-Reply-To: References: <4F48535F.6070803@gmail.com> Message-ID: Hi, > The docs regarding the mutable copying problem are very misleading. > Following Python [1] and Boost [2] scripts do not produce the same results: > > > [1] > b = a = [] > c = list(a) > a.append("s1") > print a.count("s1"), b.count("s1"), c.count("s1") #prints 1 1 0 > > [2] > list a, b, c; > b = a; > c = list(a); > a.append("s1"); > list d = extract(a); > printf("%i %i %i %i\n", a.count("s1"), b.count("s1"), c.count("s1"), > d.count("s1")); //prints 1 1 1 1 > > In [2] expected was 1 1 0 1 according to Pythonic behaviour (it states > in docs that such behaviour is mimicked). This is in conflict with > example provided in the tutorial. Ok, so it seems behaviour is different when dealing with boost::python::object vs PyObject* as constructor arguments. Using extract<> to modify mutable objects is only necessary when dealing with PyObject*: // boost_dict.cpp /* build cmdline: /apps/local/gcc/4.6.1/bin/g++ -g -Wall -R /apps/local/gcc/4.6.1/lib/ -R /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/ -L /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/ -I /var/tmp/boost/gcc/boost_1_47_0/ -I /apps/local/gcc/4.6.1/include/python2.7/ -L /apps/local/gcc/4.6.1/lib boost_dict.cpp -lboost_python -lrt -lpython2.7 */ #include #include namespace bp = boost::python; int main(void) { // otherwise we core dump Py_Initialize(); bp::object value(1); std::cout << "arg is bp::dict:" << std::endl; { bp::dict d1, d2, d3; d2 = d1; d3 = bp::dict(d1); d1["k1"] = value; bp::dict d4 = bp::extract(d1); std::cout << d1.has_key("k1") << " " << d2.has_key("k1") << " " << d3.has_key("k1") << " " << d4.has_key("k1") << std::endl; } std::cout << std::endl; std::cout << "arg is bp::handle<(PyObj*)" << std::endl; { bp::dict d1, d2, d3; // must use a handle to feed PyObj* to bp::objects bp::handle<> dict_pyobj_handle(d1.ptr()); // Note: no operator= for handle<> d2 = bp::dict(dict_pyobj_handle); d3 = bp::dict(dict_pyobj_handle); d1["k1"] = value; bp::dict d4 = bp::extract(d1.ptr()); std::cout << d1.has_key("k1") << " " << d2.has_key("k1") << " " << d3.has_key("k1") << " " << d4.has_key("k1") << std::endl; } Py_Finalize(); return 0; } $ ./a.out arg is bp::dict: 1 1 1 1 arg is bp::handle<(PyObj*) 1 0 0 1 So I think the tutorial example (if that's what you meant?) " C++: dict d(x.attr("__dict__")); // copies x.__dict__ d['whatever'] = 3; // modifies the copy " is pretty subtle, as - if I understand correctly - this - retrieves an attribute proxy from x - that returns the bp::object-wrapped __dict__ attribute (which copies the original PyObject* __dict__) - constructs d from this new bp::object (which increases the refcount but doesn't make another copy) and the assignment operates on this new bp::object. Regards Holger Landesbank Baden-Wuerttemberg Anstalt des oeffentlichen Rechts Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz HRA 12704 Amtsgericht Stuttgart From wmamrak at gmail.com Tue Feb 28 21:57:41 2012 From: wmamrak at gmail.com (Wojciech Mamrak) Date: Tue, 28 Feb 2012 21:57:41 +0100 Subject: [C++-sig] how to override __setattr__ of an extension class In-Reply-To: References: <4F48535F.6070803@gmail.com> Message-ID: Yes, this is what I meant :) I will precise my previous message. The docs are very concise on some topics, hence misleading. If you find such statement: "Beware the common pitfall of forgetting that the constructors of most of Python's mutable types make copies, just as in Python." then it is confusing why: dict a; dict b = dict(a); //copy? in Python yes, here not b["key"] = 1; (a.has_key("key")) ? puts("KEY FOUND") : puts("KEY NOT FOUND"); //KEY FOUND yields KEY FOUND, opposite to its Python equivalent (posted earlier). But the docs are correct, since they provide more or less such exemplary (and different) code: object x = import("__main__"); dict a(x.attr("__dict__")); //copy dict b = extract(x.attr("__dict__")); //no copy x.attr("__dict__")["key"] = 1; (a.has_key("key")) ? puts("KEY FOUND") : puts("KEY NOT FOUND"); //KEY NOT FOUND (b.has_key("key")) ? puts("KEY FOUND") : puts("KEY NOT FOUND"); //KEY FOUND regards 2012/2/28 Holger Joukl : > Hi, > >> The docs regarding the mutable copying problem are very misleading. >> Following Python [1] and Boost [2] scripts do not produce the same > results: >> >> >> [1] >> b = a = [] >> c = list(a) >> a.append("s1") >> print a.count("s1"), b.count("s1"), c.count("s1") #prints 1 1 0 >> >> [2] >> list a, b, c; >> b = a; >> c = list(a); >> a.append("s1"); >> list d = extract(a); >> printf("%i %i %i %i\n", a.count("s1"), b.count("s1"), c.count("s1"), >> d.count("s1")); //prints 1 1 1 1 >> >> In [2] expected was 1 1 0 1 according to Pythonic behaviour (it states >> in docs that such behaviour is mimicked). This is in conflict with >> example provided in the tutorial. > > Ok, so it seems behaviour is different when dealing with > boost::python::object vs > PyObject* as constructor arguments. > > Using extract<> to modify mutable objects is only necessary when dealing > with PyObject*: > > // boost_dict.cpp > /* build cmdline: > > /apps/local/gcc/4.6.1/bin/g++ -g -Wall -R /apps/local/gcc/4.6.1/lib/ > -R /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/ > -L /var/tmp/boost/gcc/boost_1_47_0/stage/gcc-4.6.1/py2.7/boost/1.47.0/lib/ > -I /var/tmp/boost/gcc/boost_1_47_0/ > -I /apps/local/gcc/4.6.1/include/python2.7/ ?-L /apps/local/gcc/4.6.1/lib > boost_dict.cpp -lboost_python -lrt -lpython2.7 > */ > > #include > #include > > namespace bp = boost::python; > > > int main(void) { > ? ?// otherwise we core dump > ? ?Py_Initialize(); > > ? ?bp::object value(1); > > ? ?std::cout << "arg is bp::dict:" << std::endl; > ? ?{ > ? ? ? ?bp::dict d1, d2, d3; > ? ? ? ?d2 = d1; > ? ? ? ?d3 = bp::dict(d1); > ? ? ? ?d1["k1"] = value; > ? ? ? ?bp::dict d4 = bp::extract(d1); > > ? ? ? ?std::cout << d1.has_key("k1") > ? ? ? ? ? ? ? ? ?<< " " << d2.has_key("k1") > ? ? ? ? ? ? ? ? ?<< " " << d3.has_key("k1") > ? ? ? ? ? ? ? ? ?<< " " << d4.has_key("k1") << std::endl; > ? ?} > > ? ?std::cout << std::endl; > > ? ?std::cout << "arg is bp::handle<(PyObj*)" << std::endl; > ? ?{ > ? ? ? ?bp::dict d1, d2, d3; > > ? ? ? ?// must use a handle to feed PyObj* to bp::objects > ? ? ? ?bp::handle<> dict_pyobj_handle(d1.ptr()); > > ? ? ? ?// Note: no operator= for handle<> > ? ? ? ?d2 = bp::dict(dict_pyobj_handle); > ? ? ? ?d3 = bp::dict(dict_pyobj_handle); > ? ? ? ?d1["k1"] = value; > ? ? ? ?bp::dict d4 = bp::extract(d1.ptr()); > ? ? ? ?std::cout << d1.has_key("k1") > ? ? ? ? ? ? ? ? ?<< " " << d2.has_key("k1") > ? ? ? ? ? ? ? ? ?<< " " << d3.has_key("k1") > ? ? ? ? ? ? ? ? ?<< " " << d4.has_key("k1") << std::endl; > ? ?} > > > ? ?Py_Finalize(); > ? ?return 0; > } > > > $ ./a.out > arg is bp::dict: > 1 1 1 1 > > arg is bp::handle<(PyObj*) > 1 0 0 1 > > So I think the tutorial example (if that's what you meant?) > " > C++: > > > dict d(x.attr("__dict__")); ?// copies x.__dict__ > d['whatever'] = 3; ? ? ? ? ? // modifies the copy > " > > > is pretty subtle, as - if I understand correctly - this > > > - retrieves an attribute proxy from x > > > - that returns the bp::object-wrapped __dict__ attribute (which copies the > original PyObject* __dict__) > > > - constructs d from this new bp::object (which increases the refcount but > doesn't make another copy) > > > and the assignment operates on this new bp::object. > > Regards > Holger > > Landesbank Baden-Wuerttemberg > Anstalt des oeffentlichen Rechts > Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz > HRA 12704 > Amtsgericht Stuttgart > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig From dave at boostpro.com Tue Feb 28 22:18:21 2012 From: dave at boostpro.com (Dave Abrahams) Date: Tue, 28 Feb 2012 16:18:21 -0500 Subject: [C++-sig] Building Boost.Python against a debug build of Python References: Message-ID: Please read: http://www.boost.org/doc/libs/1_49_0/libs/python/doc/building.html#python-debugging-builds HTH, Dave on Wed Feb 15 2012, Adam Preble wrote: > I previously posted about runtime oddness when switching to a debug > build of python. ?I figured I needed to rebuild Boost.Python. ?The > build process is fixated on using the normal python.exe and python.lib > paths, rather than python_d.exe and python_d.lib. ?I have put those > files in my Python26 directory, and then tried to get fancy by > renaming them to their release names instead. ?It gets mad at me . . . > > > ?Creating library > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture- > x86\debug-store-database\debug-symbols-on\python-debugging-on\threading- > multi\boost_python-vc100-mt-y-1_47.lib and object > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture- > x86\debug-store-database\debug-symbols-on\python-debugging-on\threading- > multi\boost_python-vc100-mt-y-1_47.exp > module.obj : error LNK2019: unresolved external symbol > __imp__Py_InitModule4 referenced in function "struct _object * __cdecl > boost::python::detail::init_module(char const *,void (__cdecl*) > (void))" (?init_module at detail@python at boost@@YAPAU_object@@PBDP6AXXZ at Z) > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture- > x86\debug-store-database\debug-symbols-on\python-debugging-on\threading- > multi\boost_python-vc100-mt-y-1_47.dll : fatal error LNK1120: 1 > unresolved externals > > I'm thinking that isn't what I'm supposed to do anyways. ?Is there > something I need to do in my build string? ?This is how I would do a > full build. ?Note all the baggage I added for building Python: > > [FWIW I'm building Boost 1.47 with VS2010's toolchain] > > set INCLUDE=D:\Python26\include;D:\Python26\include\Stackless > b2 --toolset=msvc-10.0 --build-type=complete stage > b2 --toolset=msvc-10.0 --build-type=complete architecture=x86 > address-model=32 stage > b2 --toolset=msvc-10.0 --with-python --build-type=complete > --debug-configuration architecture=x86 address-model=32 stage > debug-symbols=on debug-store=database python-debugging=on > > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig -- Dave Abrahams BoostPro Computing http://www.boostpro.com From dave at boostpro.com Tue Feb 28 22:06:36 2012 From: dave at boostpro.com (Dave Abrahams) Date: Tue, 28 Feb 2012 16:06:36 -0500 Subject: [C++-sig] boost python & context manager/with statement References: Message-ID: on Sun Feb 12 2012, Avi Bahra wrote: > Does boost python support context management using the with > statement? In my case I need to manage __enter__/__exit__ on > the c++ side. Could not find any examples for this in the > documentation. > > Any help appreciated. > > Best regards, > Ta, > ? ? Avi There's no explicit support for it, no. Sorry, -- Dave Abrahams BoostPro Computing http://www.boostpro.com From apt.buza at gmail.com Wed Feb 29 00:07:24 2012 From: apt.buza at gmail.com (Andrew Buza) Date: Tue, 28 Feb 2012 15:07:24 -0800 Subject: [C++-sig] Boost.Python 1.33.1 and static __doc__ properties Message-ID: I have a set of libraries that I've been trying to build for Python 2.7 using the 1.33.1 release of Boost.Python. I've used them in the past with Python 2.4, but Py2.6 made the __doc__ property on classes read-only. Because of this change I get an AttributeError (with the message "'Boost.Python.StaticProperty' object attribute '__doc__' is read-only") when I try to import the module. There's another post to this mailing list about this issue at http://mail.python.org/pipermail/cplusplus-sig/2009-August/014747.html but it doesn't look like it was ever resolved. I've tried removing explicit docstrings from my source to see if I could avoid that but things like .def_readonly appear to add them implicitly. Updating to a newer Boost release is the obvious answer but that's extremely painful for a number of reasons, one them being a handful of patches that have been applied to the my local Boost 1.33.1 release that I'd need to port to the latest version. I'd like to modify Boost.Python 1.33.1 to avoid setting static properties and I've made a few attempts at that but I haven't been successful so far. Could someone more familiar with the code tell me if this is a reasonably simple modification and maybe point me in the right direction? -------------- next part -------------- An HTML attachment was scrubbed... URL: From adam.preble at gmail.com Wed Feb 29 06:11:05 2012 From: adam.preble at gmail.com (Adam Preble) Date: Tue, 28 Feb 2012 23:11:05 -0600 Subject: [C++-sig] Building Boost.Python against a debug build of Python In-Reply-To: References: Message-ID: Too late--I managed to get it to build a bit ago. But that being said, I had read that little blurb while trying to figure out what was going on. Unfortunately it's the kind of thing that makes sense if you already know what to do, but it didn't make any sense to me when I was stammering through the process. That happens all the time and I'm sure I have plenty of stuff just like that lying around too. I'll probably post some blog thing somewhere on personal hosting and get it harvested onto Google soon, but in the meantime, I figure it would be best for the list to briefly go over the building process for those that just use Boost.Build enough to build Boost: 1. Build a debug python. 2. Develop a Boost.Build user config to select the debug python and build against it, in a way consistent with your other already-built Boost stuff (see example). 3. Run the bjam command against the user config to build Boost.Python for debugging. 4. Rebuild your target project, enabling BOOST_DEBUG_PYTHON, and potentially BOOST_PYTHON_NO_LIB (that was what killed me). For me, the user config: using python : 2.6 : D:\\temp\\stackless_src\\python-2.6.5-stackless\\PCbuild\\python_d.exe : D:\\Python26\\Include : D:\\temp\\stackless_src\\python-2.6.5-stackless\\PCbuild : on ; And the bjam command: bjam --user-config=user-config.jam --with-python python-debugging=on threading=multi variant=debug-python link=static stage Granted, there's probably been, like, five of us that have ever wanted to do this. On Tue, Feb 28, 2012 at 3:18 PM, Dave Abrahams wrote: > > Please read: > > http://www.boost.org/doc/libs/1_49_0/libs/python/doc/building.html#python-debugging-builds > > HTH, > Dave > > on Wed Feb 15 2012, Adam Preble wrote: > > > I previously posted about runtime oddness when switching to a debug > > build of python. I figured I needed to rebuild Boost.Python. The > > build process is fixated on using the normal python.exe and python.lib > > paths, rather than python_d.exe and python_d.lib. I have put those > > files in my Python26 directory, and then tried to get fancy by > > renaming them to their release names instead. It gets mad at me . . . > > > > > > Creating library > > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture- > > x86\debug-store-database\debug-symbols-on\python-debugging-on\threading- > > multi\boost_python-vc100-mt-y-1_47.lib and object > > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture- > > x86\debug-store-database\debug-symbols-on\python-debugging-on\threading- > > multi\boost_python-vc100-mt-y-1_47.exp > > module.obj : error LNK2019: unresolved external symbol > > __imp__Py_InitModule4 referenced in function "struct _object * __cdecl > > boost::python::detail::init_module(char const *,void (__cdecl*) > > (void))" (?init_module at detail@python at boost@@YAPAU_object@@PBDP6AXXZ at Z) > > bin.v2\libs\python\build\msvc-10.0\release\address-model-32\architecture- > > x86\debug-store-database\debug-symbols-on\python-debugging-on\threading- > > multi\boost_python-vc100-mt-y-1_47.dll : fatal error LNK1120: 1 > > unresolved externals > > > > I'm thinking that isn't what I'm supposed to do anyways. Is there > > something I need to do in my build string? This is how I would do a > > full build. Note all the baggage I added for building Python: > > > > [FWIW I'm building Boost 1.47 with VS2010's toolchain] > > > > set INCLUDE=D:\Python26\include;D:\Python26\include\Stackless > > b2 --toolset=msvc-10.0 --build-type=complete stage > > b2 --toolset=msvc-10.0 --build-type=complete architecture=x86 > > address-model=32 stage > > b2 --toolset=msvc-10.0 --with-python --build-type=complete > > --debug-configuration architecture=x86 address-model=32 stage > > debug-symbols=on debug-store=database python-debugging=on > > > > > > > > _______________________________________________ > > Cplusplus-sig mailing list > > Cplusplus-sig at python.org > > http://mail.python.org/mailman/listinfo/cplusplus-sig > > -- > Dave Abrahams > BoostPro Computing > http://www.boostpro.com > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From johan.rade at gmail.com Wed Feb 29 09:30:07 2012 From: johan.rade at gmail.com (=?ISO-8859-1?Q?Johan_R=E5de?=) Date: Wed, 29 Feb 2012 09:30:07 +0100 Subject: [C++-sig] throw_error_already_set and compiler warnings Message-ID: The declaration BOOST_PYTHON_DECL void throw_error_already_set(); should maybe be changed to BOOST_PYTHON_DECL BOOST_ATTRIBUTE_NORETURN void throw_error_already_set(); This will help eliminate warnings on some compilers, and will save people the trouble of writing code like int f() { ... boost::python::throw_error_already_set(); return 0; // unreachable but needed to silence compiler warning } BOOST_ATTRIBUTE_NORETURN is defined in boost/exception/detail/attribute_noreturn.hpp. From trigves at yahoo.com Wed Feb 29 14:19:43 2012 From: trigves at yahoo.com (Trigve Siver) Date: Wed, 29 Feb 2012 05:19:43 -0800 (PST) Subject: [C++-sig] [Boost-users] boost python & context manager/with statement In-Reply-To: References: Message-ID: <1330521583.73932.YahooMailNeo@web110406.mail.gq1.yahoo.com> > From: Dave Abrahams > To: boost-users at lists.boost.org > Cc: cplusplus-sig at python.org > Sent: Tuesday, February 28, 2012 10:06 PM > Subject: Re: [Boost-users] boost python & context manager/with statement > > > on Sun Feb 12 2012, Avi Bahra wrote: > >> Does boost python support context management using the with >> statement?? In my case I need to manage __enter__/__exit__ on >> the c++ side. Could not find any examples for this in the >> documentation. >> >> Any help appreciated. >> >> Best regards, >> Ta, >> ? ? Avi > > There's no explicit support for it, no. > > Sorry, But I think you could implement it like this: ?class_(...) .def("__enter__",?&somefunc) .def("__exit__",?&someotherfunc) ; Trigve