From a.viamontes.esquivel at gmail.com Mon Jun 1 13:41:03 2009 From: a.viamontes.esquivel at gmail.com (Alcides Viamontes Esquivel) Date: Mon, 1 Jun 2009 07:41:03 -0400 Subject: [C++-sig] boost::python for Python 3.0 Message-ID: Dear all: I would like to know if there is any progress on the development of a version of boost::python for Python 3.0, the last activity that I can track on this is back to the first days of April; that includes mailing lists and the boost repository. I played a little with it and last night I succeeded in compiling boost::python for Python 3.0 and in running the first test (test_builtin_converters.py), but of course a lot have still to be done (the const char* and char* issue, among many ones...). I would like to know if there is somebody else working on this. Any suggestions appreciated. Alcides. From divinekid at gmail.com Mon Jun 1 14:06:20 2009 From: divinekid at gmail.com (Haoyu Bai) Date: Mon, 1 Jun 2009 20:06:20 +0800 Subject: [C++-sig] boost::python for Python 3.0 In-Reply-To: References: Message-ID: <1d7983e80906010506w15b6167al9fcab30c7adb01f7@mail.gmail.com> On Mon, Jun 1, 2009 at 7:41 PM, Alcides Viamontes Esquivel wrote: > Dear all: > > I would like to know if there is any progress on the development of a > version of boost::python for Python 3.0, the last activity that I can > track on this is back to the first days of April; that includes > mailing lists and the boost repository. > > I played a little with it and last night I succeeded in compiling > boost::python for Python 3.0 and in running the first test > (test_builtin_converters.py), but of course a lot have still to be > done (the const char* and char* issue, among many ones...). I would > like to know if there is somebody else working on this. > > Any suggestions appreciated. > > Alcides. > Hi Alcides, Yes I'm working on py3k support as my GSoC project. And as you see, the current situation is it can be compiled under Python 3, and have some tests passed. But now I get stuck on the Python unicode string and const char* conversion issue. In Python 2.x, the Python string -> const char* conversion can be simply done by a lvalue conversion. But in Python 3, the conversion path would be unicode -> (encoding to) bytes -> const char*. This requires an rvalue conversion. Bit after reading the code in extract.hpp, I found that since char* is a pointer, the conversion must be done as a lvalue conversion. This is a short description of the problem, please forgive me if it is not clear. :) So I'm still reading the code about call policies and type converting, hopefully I can state the problem clearer and even find out a solution when I get more understanding about BPL internal. Thanks for your interesting! :) -- Haoyu Bai From a.viamontes.esquivel at gmail.com Mon Jun 1 14:52:17 2009 From: a.viamontes.esquivel at gmail.com (Alcides Viamontes Esquivel) Date: Mon, 1 Jun 2009 08:52:17 -0400 Subject: [C++-sig] boost::python for Python 3.0 In-Reply-To: <1d7983e80906010506w15b6167al9fcab30c7adb01f7@mail.gmail.com> References: <1d7983e80906010506w15b6167al9fcab30c7adb01f7@mail.gmail.com> Message-ID: Good to know that you are on it! I see your point with const char*, not to mention char* (there is a 'rewrap' in test_builtin_converters.py for that too). For PyUnicode objects, lvalue conversions are not an option. Short of that, the only remaining solution is to do an rvalue conversion by encoding and putting the result in a buffer which the user is responsible of releasing, as in this: scoped_array< char > my_chars = extract< const char* >(a_python_object); .... do something with it.... The shortcomings of this are: * The encoding to use is restricted to one without zeroes (e.g. utf-8) * It is easy to leak memory, because legacy code using boost::python might have been instructed in considering my_chars a pointer to an internal reference. And it should remain that way for Python 2.x. This point breaks the approach completely. However, as you pointed out, a solution can be worked using call policies. I will think on it a little more. Regards! On Mon, Jun 1, 2009 at 8:06 AM, Haoyu Bai wrote: > On Mon, Jun 1, 2009 at 7:41 PM, Alcides Viamontes Esquivel > wrote: >> Dear all: >> >> I would like to know if there is any progress on the development of a >> version of boost::python for Python 3.0, the last activity that I can >> track on this is back to the first days of April; that includes >> mailing lists and the boost repository. >> >> I played a little with it and last night I succeeded in compiling >> boost::python for Python 3.0 and in running the first test >> (test_builtin_converters.py), but of course a lot have still to be >> done (the const char* and char* issue, among many ones...). I would >> like to know if there is somebody else working on this. >> >> Any suggestions appreciated. >> >> Alcides. >> > > Hi Alcides, > > Yes I'm working on py3k support as my GSoC project. And as you see, > the current situation is it can be compiled under Python 3, and have > some tests ?passed. > > But now I get stuck on the Python unicode string and const char* > conversion issue. In Python 2.x, the Python string -> const char* > conversion can be simply done by a lvalue conversion. But in Python 3, > the conversion path would be unicode -> (encoding to) bytes -> const > char*. This requires an rvalue conversion. Bit after reading the code > in extract.hpp, I found that since char* is a pointer, the conversion > must be done as a lvalue conversion. This is a short description of > the problem, please forgive me if it is not clear. :) > > So I'm still reading the code about call policies and type converting, > hopefully I can state the problem clearer and even find out a solution > when I get more understanding about BPL internal. > > Thanks for your interesting! :) > > -- Haoyu Bai > From mrcastle at gmail.com Mon Jun 1 16:12:53 2009 From: mrcastle at gmail.com (=?ISO-8859-1?Q?Kevin_B=E9lisle?=) Date: Mon, 1 Jun 2009 10:12:53 -0400 Subject: [C++-sig] Advantages/Disadvantages Message-ID: <2FCF4B21-9B2D-4F21-B04A-55B0A2B45FF7@gmail.com> Hello, I was wondering what were the advantages and disadvantages of using Boost.Python or SWIG as opposed to just working with the built-in features that allow the embedding/wrapping of C++ code into PYTHON projects. Will using Boost.Python or SWIG make life easier while coding but result in a loss of performance at runtime and an increase in processing time? Thanks, Kevin From blake at hailmail.net Mon Jun 1 22:38:31 2009 From: blake at hailmail.net (Dane Springmeyer) Date: Mon, 1 Jun 2009 13:38:31 -0700 Subject: [C++-sig] boost::python for Python 3.0 In-Reply-To: References: <1d7983e80906010506w15b6167al9fcab30c7adb01f7@mail.gmail.com> Message-ID: <5F24EA71-E85D-49BB-814B-3F437ED1759D@hailmail.net> On Jun 1, 2009, at 5:52 AM, Alcides Viamontes Esquivel wrote: > Good to know that you are on it! Same here, I'm super excited about python 3 compatibility! I work on http://mapnik.org which uses boost python heavily. Cheers, Dane From aamir.nedian at gmail.com Tue Jun 2 17:16:00 2009 From: aamir.nedian at gmail.com (Aamir Ahmed Khan) Date: Tue, 2 Jun 2009 17:16:00 +0200 Subject: [C++-sig] How to pass pointer to an array in Python for a wrapped C++ function Message-ID: I am new to C++/Python mixed language programming and do not have much idea about Python/C API. I just started using Boost.Python to wrap a C++ library for Python. I am stuck at wrapping a function that takes pointer to an array as an argument. Following (2nd ctor) is its prototype in C++. class AAF{ AAF(AAF_TYPE t); AAF(double v0, const double * t1, const unsigned * t2, unsigned T); ~AAF(); } Am I doing right by wrapping it like this in boost::python? class_("AAF", init()) .def(init()); Note that it compiled and linked successfully, but I couldn't figure out how to call it in Python. My naive tries like the following failed. >>> z = AAF(10, [4, 5.5, 10], [1, 1, 2], 3); Traceback (most recent call last): File "./test_interval.py", line 40, in z = AAF(10, [4, 5.5, 10], [1, 1, 2], 3); Boost.Python.ArgumentError: Python argument types in AAF.__init__(AAF, int, list, list, int) did not match C++ signature: __init__(_object*, AAF_TYPE) __init__(_object*, double, double const*, unsigned int const*, unsigned int) >>> t1 = array.array('d', [4, 5.5, 10]) >>> t2 = array.array('I', [1, 1, 2]) >>> z = AAF(10, t1, t2, 3); Traceback (most recent call last): File "./test_interval.py", line 40, in z = AAF(10, t1, t2, 3); Boost.Python.ArgumentError: Python argument types in AAF.__init__(AAF, int, array.array, array.array, int) did not match C++ signature: __init__(_object*, AAF_TYPE) __init__(_object*, double, double const*, unsigned int const*, unsigned int) My second question is that do I also need to wrap the destructor? Please specify if this might be necessary in some cases but not always. -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan.seefeld at gmail.com Mon Jun 1 14:04:21 2009 From: stefan.seefeld at gmail.com (Stefan Seefeld) Date: Mon, 01 Jun 2009 08:04:21 -0400 Subject: [C++-sig] boost::python for Python 3.0 In-Reply-To: References: Message-ID: <4A23C3C5.60707@gmail.com> Alcides Viamontes Esquivel wrote: > Dear all: > > I would like to know if there is any progress on the development of a > version of boost::python for Python 3.0, the last activity that I can > track on this is back to the first days of April; that includes > mailing lists and the boost repository. > Python 3 support is being worked on as a Google Summer of Code project. To track status, you may want to look at the development branch in https://svn.boost.org/svn/boost/sandbox-branches/bhy/py3k or join the occasional discussions on IRC over irc://irc.freenode.net/boostpython Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From mikeowen at llnl.gov Thu Jun 4 02:22:22 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Wed, 3 Jun 2009 17:22:22 -0700 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? Message-ID: <41A3D5F3-384C-4BFF-B9E1-B1CDCD8DBF25@llnl.gov> I've been looking at pybindgen and have a simple question -- why do the container wrappers not implement more of the python container interface? For instance, you can expose the std::vector like so: vecint = mod.add_container("std::vector", "int", "vector", custom_name="vector_of_int") The resulting type won't let you take the length with "len" or index into the instances with []. I would naively think maybe I could add these methods as appropriate to types using "add_method" like the following: vecint.add_method("size", "int", [], custom_name = "__len__") but because "Container" does not inherit from "CppClass" I don't see how to easily extend the container in this manner. Is there a deep reason these things are this way? Thanks! Mike. From gjcarneiro at gmail.com Thu Jun 4 21:57:01 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Thu, 4 Jun 2009 20:57:01 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: <41A3D5F3-384C-4BFF-B9E1-B1CDCD8DBF25@llnl.gov> References: <41A3D5F3-384C-4BFF-B9E1-B1CDCD8DBF25@llnl.gov> Message-ID: 2009/6/4 J. Michael Owen > I've been looking at pybindgen and have a simple question -- why do the > container wrappers not implement more of the python container interface? > For instance, you can expose the std::vector like so: > > vecint = mod.add_container("std::vector", "int", "vector", > custom_name="vector_of_int") > > The resulting type won't let you take the length with "len" or index into > the instances with []. I would naively think maybe I could add these > methods as appropriate to types using "add_method" like the following: > > vecint.add_method("size", "int", [], custom_name = "__len__") > > but because "Container" does not inherit from "CppClass" I don't see how to > easily extend the container in this manner. Is there a deep reason these > things are this way? No deep reason, just lazyness. PyBindGen needs to implement some extra slots [1]. As a workaround for now, just convert the vector to a python list firtst, i.e. list(vector_object). Then you can call len() on the list instead. Not pretty but effective. [1] http://docs.python.org/c-api/typeobj.html#tp_as_sequence -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From nicolas.regnault at lpnhe.in2p3.fr Thu Jun 4 22:37:59 2009 From: nicolas.regnault at lpnhe.in2p3.fr (Nicolas Regnault) Date: Thu, 4 Jun 2009 22:37:59 +0200 (CEST) Subject: [C++-sig] const shared_ptr and py++ / boost.python Message-ID: Dear all: I am wrapping a library which relies heavily on boost smart pointers (mainly boost::shared_ptr< T >). In particular, many functions and methods have signatures like: void my_func( const boost::shared_ptr< MyClass > & ); Everything works well from within C++ programs. But I encounter a serious problem with the wrappers I have generated with py++. Indeed, when passing from python, something which wraps a "shared_ptr< MyClass >" to my_function(), the code within the function apparently gets a shared_ptr with a use_count equal to 1 (pointing to the same address), whatever the use_count of the shared_ptr passed to the function. I haven't been able to see what boost.python is doing in between, but something gets altered. Any idea of what is wrong with the wrappers ? Here is a small example: //////////////////////////////////////////////////////////////////////// // -*- C++ -*- // //tst.hpp // #ifndef TST_HPP #define TST_HPP #include #include struct Foo { Foo() : count(0) {} int count; }; typedef std::vector< boost::shared_ptr< Foo > > FooVector; void build_vector( boost::shared_ptr< Foo > &, FooVector & ); // print the shared_ptr<>::use_count void print_use_count( boost::shared_ptr< Foo > & ); // print the shared_ptr<>::use_count void print_use_count_const( const boost::shared_ptr< Foo > & ); #endif //////////////////////////////////////////////////////////////////////// // -*- C++ -*- // // tst.cpp // #include #include #include "tst.hpp" using namespace std; using namespace boost; void build_vector( shared_ptr< Foo > & foo_ptr, FooVector & vec ) { vec.clear(); for( unsigned int i=0; i<100; i++ ) { vec.push_back( foo_ptr ); } } void print_use_count( shared_ptr< Foo > & ptr ) { cout << "[print_use_count] use_count=" << ptr.use_count() << " addr=" << std::ios::hex << ptr.get() << endl; } void print_use_count_const( const shared_ptr< Foo > & ptr ) { cout << "[print_use_count] use_count=" << ptr.use_count() << " addr=" << std::ios::hex << ptr.get() << endl; } //////////////////////////////////////////////////////////////////////// // -*- C++ -*- // // export_tst.cpp // #include "boost/shared_ptr.hpp" #include "boost/python.hpp" #include "boost/python/suite/indexing/vector_indexing_suite.hpp" #include "tst.hpp" namespace bp = boost::python; BOOST_PYTHON_MODULE(_tst){ bp::class_< std::vector< boost::shared_ptr > >("vector_Foo") .def( bp::vector_indexing_suite< ::std::vector< boost::shared_ptr >, true >() ); { //::Foo typedef bp::class_< Foo, boost::shared_ptr< Foo > > Foo_exposer_t; Foo_exposer_t Foo_exposer = Foo_exposer_t( "Foo" ); bp::scope Foo_scope( Foo_exposer ); Foo_exposer.def( bp::init< >() ); Foo_exposer.def_readwrite( "count", &Foo::count ); } { //::build_vector typedef void ( *build_vector_function_type )( ::boost::shared_ptr< Foo > &,::FooVector & ); bp::def( "build_vector" , build_vector_function_type( &::build_vector ) , ( bp::arg("arg0"), bp::arg("arg1") ) ); } { //::print_use_count typedef void ( *print_use_count_function_type )( ::boost::shared_ptr< Foo > & ); bp::def( "print_use_count" , print_use_count_function_type( &::print_use_count ) , ( bp::arg("arg0") ) ); } { //::print_use_count_const typedef void ( *print_use_count_const_function_type )( ::boost::shared_ptr< Foo > const & ); bp::def( "print_use_count_const" , print_use_count_const_function_type( &::print_use_count_const ) , ( bp::arg("arg0") ) ); } } //////////////////////////////////////////////////////////////////////// // -*- C++ -*- // // main.cpp // #include #include #include "tst.hpp" using namespace boost; int main() { shared_ptr< Foo > f( new Foo ); FooVector vec; build_vector(f, vec); print_use_count(f); print_use_count_const(f); } ######################################################################## #!/usr/bin/env python import _tst def main(): f = _tst.Foo() vec = _tst.vector_Foo() _tst.build_vector(f, vec) _tst.print_use_count(f) _tst.print_use_count_const(f) if __name__ == "__main__": main() ######################################################################## # Makefile ######################################################################## all: main _tst.so %.os : %.cpp g++ -c $< -o $@ -fPIC -I /usr/include/python2.5 libtst.so: tst.os g++ -shared $< -o $@ main: main.os libtst.so g++ $< -o $@ -L . -ltst _tst.so: export_tst.os libtst.so g++ -shared $< -o $@ -lboost_python -L . -ltst ######################################################################## $ ./main [print_use_count] use_count=101 addr=80x1617010 [print_use_count] use_count=101 addr=80x1617010 $ ./main.py [print_use_count] use_count=101 addr=80x121b2b0 [print_use_count] use_count=1 addr=80x121b2b0 Nicolas. -- Nicolas Regnault Laboratoire de Physique Nucl?aire et de Hautes Energies 4, place Jussieu - Tour 43 RdC - 75252 Paris Cedex 05 +33 1 44 27 73 29 From mikeowen at llnl.gov Fri Jun 5 18:49:48 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Fri, 5 Jun 2009 09:49:48 -0700 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> Message-ID: Hi Taner, Thanks for the suggestion! I actually was trying the same solution, except I wasn't sure how to handle the iterators. I'll try it out as you do in your example -- are you able to iterate over the vector in python the usual way? As in "for x in vec:"? Two curious things I noted about wrapping "std::vector" myself: 1. I can't seem to get indexing access to work, even though I wrap an indexing operation with the python name "__getitem__" 2. I also can't get python's len function to work, even if I provide "__len__" as follows: veci.add_method("size", "int", [], custom_name="__len__") Have you been able to make either of those methods work? Thanks! Mike. On Jun 4, 2009, at 11:42 PM, Taner Yildirim wrote: > Dear Mike, > > Concerning your question about add_method for stl_vector, I was able > to add > almost all standard methods of std_vector by simply wrapping the > std::vector without > even using the mod.add_container!! Since I am very new to pybindgen, > I am not sure if there is anything wrong with this approach but it > seems to work quite well. > Hopefully Gustava can comment on this more: > > > In summary, here?s how I get the std::vector work in python > using pybindgen without add_container: > > >>>>>>>>> THIS is the header file: tst1.h <<<<<<<<< > > #include > #include > typedef std::vector VecI; > typedef std::vector::iterator Iter_VecI; > #here you can do typdef for other types like double, string, etc > >>>>>>> END OF THE HEADER FILE <<<<<<<<<<<<<<<<<<<< > > > And here's the pybind-gen code to generate the wrapper.cpp > > >>>>>>>>>>>>>> wrap.py <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > import sys,pybindgen > from pybindgen import ReturnValue, Parameter, Module, Function, > FileCodeSink > > def my_module_gen(file_out): > mod=Module('tst1') > mod.add_include('"tst1.h"') > iter_veci=mod.add_class('Iter_VecI') > # > veci=mod.add_class('VecI') > veci.add_constructor([]) > > veci > .add_constructor > ([Parameter.new('int','size'),Parameter.new('int','val' )]) > veci.add_constructor([Parameter.new('VecI','vec_int')]) > veci.add_method('begin','Iter_VecI',[]) > veci.add_method('end','Iter_VecI',[]) > > veci > .add_constructor > ([Parameter > .new('Iter_VecI','begin'),Parameter.new('Iter_VecI','end')]) > veci.add_method('push_back','void', > [Parameter.new('int','value')]) > veci.add_method('size',ReturnValue.new('int'),[]) > veci.add_method('pop_back','void',[]) > veci.add_method('at',ReturnValue.new('int'), > [Parameter.new('int','index')]) > veci.add_method('front',ReturnValue.new('int'),[]) > veci.add_method('back',ReturnValue.new('int'),[]) > veci.add_method('clear','void',[]) > veci.add_method('empty',ReturnValue.new('bool'), > [],is_const=True) > veci.add_binary_comparison_operator('==') > mod.generate(file_out) > > my_module_gen(sys.stdout) > > >>>>>>>>>>> END OF THE WRAPPER CODE <<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > Here?s a simple shell script to generate the code > (I generate pyd since I work in mingw envirentment): > > >>>>>>>>> simple shell script to generate the library <<<<<<<<< > rm tst1.pyd wrap.cpp > python wrap.py > wrap.cpp > g++ -c wrap.cpp -IC:/Python25/include > g++ -shared -o tst1.pyd wrap.o -LC:/Python25/libs -lpython25 > # > >>>>>>>>>>> end of the script <<<<<<<<<< > > Here?s the usage of the tst1.pyd in python: > > >>> from tst1 import * > >>> dir() > ['Iter_VecI', 'VecI', '__builtins__', '__doc__', '__name__'] > >>> a=VecI() # default constructor > >>> a.size() # check the size > 0 > >>> a.push_back(2) # add an entry > >>> a.size() > 1 > >>> a.at(0) # get the value at entry index 0 > 2 > >>> a.push_back(2) > >>> a.push_back(3) > >>> a.size() > 3 > >>> b=VecI(a.begin(),a.end()) #constructor using iterator > >>> b.size() > 3 > >>> a==b # check if they are the same > True > >>> b.push_back(1) > >>> b.size() > 4 > >>> a==b > False > >>> b.clear() > >>> b=VecI(4,10) # constructor for 4 entry with value of 10 > >>> > >>> c=VecI(a) #contructor from an other vector > >>> c==a > True > >>> c.pop_back() > >>> c==a > False > >>> a.empty() > False > >>> a.clear() > >>> a.empty() > True > >>> > >>>>>>>>>>> SOME COMMENTS <<<<<<<<<<<<<<<<<<<<<<<<< > > Needless to say, one can simple change the type ?int? in the wrapper > code to double, float, etc and everything works for that type. > Ideally, I would like to have this sort of wrapper header file > included in pybindgen for a generic type TTTT and then we can simply > specify what type TTT we want (like TTT=[?int?,?float?, ?string?]) > and then pybindgen could just automatically generate the each > template with the requested type in the wrapper code!!! Anyway, this > would be my solution until the pybindgen container will have all the > standards methods implemented!!! > > QUESTION: In this approach, Is it possible to add a custom > constructor (in the python end) > such as a=VecI([a python list of integer])??? And how do I get a > python list from my VecI object using list?? > I guess implementing these two things are not easy and probably > that?s why we have the > pybindgen-container module at first place!!! > > Anyway, hope what I wrote above is not a total non-sense (which is > quite possible considering I have only a week experience wit > pybindgen!!!). > > Best regards > Taner > > ******************************************************************************** > Dr. Taner Yildirim, Ph.D. Physicist, > Adjunct Associate Professor, > Materials Science and Engineering, > University of Pennsylvania > and > Computational and Neutron Science Team Leader, > NIST Center for Neutron Research, > National Institute of Standards and Technology, > 100 Bureau Drive, Gaithersburg, MD 20899-6100. > Email: taner at nist.gov or taner at seas.upenn.edu > PHONE: 301-975-6228 FAX : 301-921-9847 > Web : http:// webster.ncnr.nist.gov/staff/taner > ******************************************************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gjcarneiro at gmail.com Fri Jun 5 18:57:41 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Fri, 5 Jun 2009 17:57:41 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> Message-ID: 2009/6/5 Taner Yildirim > Dear Mike, > > > > Concerning your question about add_method for stl_vector, I was able to add > > > almost all standard methods of std_vector by simply wrapping the > std::vector without > > even using the mod.add_container!! Since I am very new to pybindgen, > > I am not sure if there is anything wrong with this approach but it seems to > work quite well. > > Hopefully Gustava can comment on this more: > > > I think it is wonderful that you could get it to work without add_container. Nothing like discovering a tool you develop is being used in new and interesting ways :-) There are advantages and disadvantages to your method. On the plus side, you can wrap normal container methods, and have a more rich container with lots of methods. On the other hand, normal C++ classes do not support the iteration protocol (tp_iter), and is more work than should be needed to support this (having to declare the iterator type as well...). Also, with add_container, parameters of that type will accept also python lists in lieu of the container type itself. One possible solution that I might try implementing some day is to converge the two solutions, i.e., give iteration powers to normal C++ classes. But at the time I thought it was not very simple to do, and I opted to completely separate the two types. > > In summary, here?s how I get the std::vector work in python > > using pybindgen without add_container: > > > > >>>>>>>>> THIS is the header file: tst1.h <<<<<<<<< > > > > #include > > #include > > typedef std::vector VecI; > > typedef std::vector::iterator Iter_VecI; > > #here you can do typdef for other types like double, string, etc > > >>>>>>> END OF THE HEADER FILE <<<<<<<<<<<<<<<<<<<< > > > > > > And here's the pybind-gen code to generate the wrapper.cpp > > > > >>>>>>>>>>>>>> wrap.py <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > > > import sys,pybindgen > > from pybindgen import ReturnValue, Parameter, Module, Function, > FileCodeSink > > > > def my_module_gen(file_out): > > mod=Module('tst1') > > mod.add_include('"tst1.h"') > > iter_veci=mod.add_class('Iter_VecI') > > # > > veci=mod.add_class('VecI') > > veci.add_constructor([]) > > > veci.add_constructor([Parameter.new('int','size'),Parameter.new('int','val' > )]) > > veci.add_constructor([Parameter.new('VecI','vec_int')]) > > veci.add_method('begin','Iter_VecI',[]) > > veci.add_method('end','Iter_VecI',[]) > > > veci.add_constructor([Parameter.new('Iter_VecI','begin'),Parameter.new('Iter_VecI','end')]) > > veci.add_method('push_back','void',[Parameter.new('int','value')]) > > veci.add_method('size',ReturnValue.new('int'),[]) > > veci.add_method('pop_back','void',[]) > > > veci.add_method('at',ReturnValue.new('int'),[Parameter.new('int','index')]) > > veci.add_method('front',ReturnValue.new('int'),[]) > > veci.add_method('back',ReturnValue.new('int'),[]) > > veci.add_method('clear','void',[]) > > veci.add_method('empty',ReturnValue.new('bool'),[],is_const=True) > > veci.add_binary_comparison_operator('==') > > mod.generate(file_out) > > > > my_module_gen(sys.stdout) > > > > >>>>>>>>>>> END OF THE WRAPPER CODE <<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > > > Here?s a simple shell script to generate the code > > (I generate pyd since I work in mingw envirentment): > > > > >>>>>>>>> simple shell script to generate the library <<<<<<<<< > > rm tst1.pyd wrap.cpp > > python wrap.py > wrap.cpp > > g++ -c wrap.cpp -IC:/Python25/include > > g++ -shared -o tst1.pyd wrap.o -LC:/Python25/libs -lpython25 > > # > > >>>>>>>>>>> end of the script <<<<<<<<<< > > > > Here?s the usage of the tst1.pyd in python: > > > > >>> from tst1 import * > > >>> dir() > > ['Iter_VecI', 'VecI', '__builtins__', '__doc__', '__name__'] > > >>> a=VecI() # default constructor > > >>> a.size() # check the size > > 0 > > >>> a.push_back(2) # add an entry > > >>> a.size() > > 1 > > >>> a.at(0) # get the value at entry index 0 > > 2 > > >>> a.push_back(2) > > >>> a.push_back(3) > > >>> a.size() > > 3 > > >>> b=VecI(a.begin(),a.end()) #constructor using iterator > > >>> b.size() > > 3 > > >>> a==b # check if they are the same > > True > > >>> b.push_back(1) > > >>> b.size() > > 4 > > >>> a==b > > False > > >>> b.clear() > > >>> b=VecI(4,10) # constructor for 4 entry with value of 10 > > >>> > > >>> c=VecI(a) #contructor from an other vector > > >>> c==a > > True > > >>> c.pop_back() > > >>> c==a > > False > > >>> a.empty() > > False > > >>> a.clear() > > >>> a.empty() > > True > > >>> > > >>>>>>>>>>> SOME COMMENTS <<<<<<<<<<<<<<<<<<<<<<<<< > > > > Needless to say, one can simple change the type ?int? in the wrapper code > to double, float, etc and everything works for that type. Ideally, I would > like to have this sort of wrapper header file included in pybindgen for a > generic type TTTT and then we can simply specify what type TTT we want (like > TTT=[?int?,?float?, ?string?]) and then pybindgen could just automatically > generate the each template with the requested type in the wrapper code!!! > Anyway, this would be my solution until the pybindgen container will have > all the standards methods implemented!!! > > > > QUESTION: In this approach, Is it possible to add a custom constructor (in > the python end) > > such as a=VecI([a python list of integer])??? And how do I get a python > list from my VecI object using list?? > > I guess implementing these two things are not easy and probably that?s why > we have the > > pybindgen-container module at first place!!! > > > > Anyway, hope what I wrote above is not a total non-sense (which is quite > possible considering I have only a week experience wit pybindgen!!!). > > > > Best regards > > Taner > > > > > ******************************************************************************** > > Dr. Taner Yildirim, Ph.D. Physicist, > > Adjunct Associate Professor, > > Materials Science and Engineering, > > University of Pennsylvania > > and > > Computational and Neutron Science Team Leader, > > NIST Center for Neutron Research, > > National Institute of Standards and Technology, > > 100 Bureau Drive, Gaithersburg, MD 20899-6100. > > Email: taner at nist.gov or taner at seas.upenn.edu > > PHONE: 301-975-6228 FAX : 301-921-9847 > > Web : http://webster.ncnr.nist.gov/staff/taner > > > ******************************************************************************** > > > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From gjcarneiro at gmail.com Fri Jun 5 19:00:55 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Fri, 5 Jun 2009 18:00:55 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> Message-ID: 2009/6/5 J. Michael Owen > Hi Taner, > Thanks for the suggestion! I actually was trying the same solution, except > I wasn't sure how to handle the iterators. I'll try it out as you do in > your example -- are you able to iterate over the vector in python the usual > way? As in "for x in vec:"? > > Two curious things I noted about wrapping "std::vector" myself: > > 1. I can't seem to get indexing access to work, even though I wrap an > indexing operation with the python name "__getitem__" > > 2. I also can't get python's len function to work, even if I provide > "__len__" as follows: > veci.add_method("size", "int", [], custom_name="__len__") > > Have you been able to make either of those methods work? > It won't work because those are special methods in C/C++ side; Python extension types use a "slots" mechanism for this kind of thing, see http://www.python.org/doc/2.6/c-api/typeobj.html > > Thanks! > > Mike. > > On Jun 4, 2009, at 11:42 PM, Taner Yildirim wrote: > > Dear Mike, > > Concerning your question about add_method for stl_vector, I was able to add > almost all standard methods of std_vector by simply wrapping the > std::vector without > even using the mod.add_container!! Since I am very new to pybindgen, > I am not sure if there is anything wrong with this approach but it seems to > work quite well. > Hopefully Gustava can comment on this more: > > > In summary, here?s how I get the std::vector work in python > using pybindgen without add_container: > > >>>>>>>>> THIS is the header file: tst1.h <<<<<<<<< > > #include > #include > typedef std::vector VecI; > typedef std::vector::iterator Iter_VecI; > #here you can do typdef for other types like double, string, etc > >>>>>>> END OF THE HEADER FILE <<<<<<<<<<<<<<<<<<<< > > > And here's the pybind-gen code to generate the wrapper.cpp > > >>>>>>>>>>>>>> wrap.py <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > import sys,pybindgen > from pybindgen import ReturnValue, Parameter, Module, Function, > FileCodeSink > > def my_module_gen(file_out): > mod=Module('tst1') > mod.add_include('"tst1.h"') > iter_veci=mod.add_class('Iter_VecI') > # > veci=mod.add_class('VecI') > veci.add_constructor([]) > > veci.add_constructor([Parameter.new('int','size'),Parameter.new('int','val' > )]) > veci.add_constructor([Parameter.new('VecI','vec_int')]) > veci.add_method('begin','Iter_VecI',[]) > veci.add_method('end','Iter_VecI',[]) > > veci.add_constructor([Parameter.new('Iter_VecI','begin'),Parameter.new('Iter_VecI','end')]) > veci.add_method('push_back','void',[Parameter.new('int','value')]) > veci.add_method('size',ReturnValue.new('int'),[]) > veci.add_method('pop_back','void',[]) > > veci.add_method('at',ReturnValue.new('int'),[Parameter.new('int','index')]) > veci.add_method('front',ReturnValue.new('int'),[]) > veci.add_method('back',ReturnValue.new('int'),[]) > veci.add_method('clear','void',[]) > veci.add_method('empty',ReturnValue.new('bool'),[],is_const=True) > veci.add_binary_comparison_operator('==') > mod.generate(file_out) > > my_module_gen(sys.stdout) > > >>>>>>>>>>> END OF THE WRAPPER CODE <<<<<<<<<<<<<<<<<<<<<<<<<<<<< > > Here?s a simple shell script to generate the code > (I generate pyd since I work in mingw envirentment): > > >>>>>>>>> simple shell script to generate the library <<<<<<<<< > rm tst1.pyd wrap.cpp > python wrap.py > wrap.cpp > g++ -c wrap.cpp -IC:/Python25/include > g++ -shared -o tst1.pyd wrap.o -LC:/Python25/libs -lpython25 > # > >>>>>>>>>>> end of the script <<<<<<<<<< > > Here?s the usage of the tst1.pyd in python: > > >>> from tst1 import * > >>> dir() > ['Iter_VecI', 'VecI', '__builtins__', '__doc__', '__name__'] > >>> a=VecI() # default constructor > >>> a.size() # check the size > 0 > >>> a.push_back(2) # add an entry > >>> a.size() > 1 > >>> a.at(0) # get the value at entry index 0 > 2 > >>> a.push_back(2) > >>> a.push_back(3) > >>> a.size() > 3 > >>> b=VecI(a.begin(),a.end()) #constructor using iterator > >>> b.size() > 3 > >>> a==b # check if they are the same > True > >>> b.push_back(1) > >>> b.size() > 4 > >>> a==b > False > >>> b.clear() > >>> b=VecI(4,10) # constructor for 4 entry with value of 10 > >>> > >>> c=VecI(a) #contructor from an other vector > >>> c==a > True > >>> c.pop_back() > >>> c==a > False > >>> a.empty() > False > >>> a.clear() > >>> a.empty() > True > >>> > >>>>>>>>>>> SOME COMMENTS <<<<<<<<<<<<<<<<<<<<<<<<< > > Needless to say, one can simple change the type ?int? in the wrapper code > to double, float, etc and everything works for that type. Ideally, I would > like to have this sort of wrapper header file included in pybindgen for a > generic type TTTT and then we can simply specify what type TTT we want (like > TTT=[?int?,?float?, ?string?]) and then pybindgen could just automatically > generate the each template with the requested type in the wrapper code!!! > Anyway, this would be my solution until the pybindgen container will have > all the standards methods implemented!!! > > QUESTION: In this approach, Is it possible to add a custom constructor (in > the python end) > such as a=VecI([a python list of integer])??? And how do I get a python > list from my VecI object using list?? > I guess implementing these two things are not easy and probably that?s why > we have the > pybindgen-container module at first place!!! > > Anyway, hope what I wrote above is not a total non-sense (which is quite > possible considering I have only a week experience wit pybindgen!!!). > > Best regards > Taner > > > ******************************************************************************** > Dr. Taner Yildirim, Ph.D. Physicist, > Adjunct Associate Professor, > Materials Science and Engineering, > University of Pennsylvania > and > Computational and Neutron Science Team Leader, > NIST Center for Neutron Research, > National Institute of Standards and Technology, > 100 Bureau Drive, Gaithersburg, MD 20899-6100. > Email: taner at nist.gov or taner at seas.upenn.edu > PHONE: 301-975-6228 FAX : 301-921-9847 > Web : http:// webster.ncnr.nist.gov/staff/taner > > ******************************************************************************** > > > > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeowen at llnl.gov Fri Jun 5 22:28:46 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Fri, 5 Jun 2009 13:28:46 -0700 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> Message-ID: <903DA42C-6F3C-4C6F-AAA6-146627934C60@llnl.gov> Hi Gustavo, Hmm. OK, if I read that page correctly what I would need in order to emulate a python sequence, mapping, or number is the ability to register PySequenceMethods, PyMappingMethods, or PyNumberMethods then? Maybe I'll dig around in the code and see if this is difficult to add. It would certainly be useful for what I'm trying to do! Mike. On Jun 5, 2009, at 10:00 AM, Gustavo Carneiro wrote: > > Two curious things I noted about wrapping "std::vector" myself: > > 1. I can't seem to get indexing access to work, even though I wrap > an indexing operation with the python name "__getitem__" > > 2. I also can't get python's len function to work, even if I > provide "__len__" as follows: > veci.add_method("size", "int", [], custom_name="__len__") > > Have you been able to make either of those methods work? > > It won't work because those are special methods in C/C++ side; > Python extension types use a "slots" mechanism for this kind of > thing, see http:// www. python.org/doc/2.6/c-api/typeobj.html > -------------- next part -------------- An HTML attachment was scrubbed... URL: From j.reid at mail.cryst.bbk.ac.uk Sat Jun 6 12:54:29 2009 From: j.reid at mail.cryst.bbk.ac.uk (John Reid) Date: Sat, 06 Jun 2009 11:54:29 +0100 Subject: [C++-sig] boost::multi_array numpy conversion Message-ID: I've developed some boost::python code that allows conversion from numpy arrays to boost::multi_array_refs and from boost::multi_array(ref)s to numpy arrays without copying data. It does not deal with lifetime management but nevertheless I've found it very useful. I know pyublas performs a similar function including lifetime management but that is restricted to matrices and vectors. So if anyone is interested I'd be very happy to share it. It does already handle both C and fortran ordered storage and I have in mind to extend it to deal with slices, strides, etc... but have not needed this yet. John. From divinekid at gmail.com Sat Jun 6 17:23:34 2009 From: divinekid at gmail.com (Haoyu Bai) Date: Sat, 6 Jun 2009 23:23:34 +0800 Subject: [C++-sig] Py3k unicode to const char* conversion solved (Re: boost::python for Python 3.0) Message-ID: <1d7983e80906060823h42d7aeeet443e651f320279a2@mail.gmail.com> Hi, After attempted all the hacks come from my mind, I solved the unicode string to const char* conversion problem by simply calling a Python C-API: _PyUnicode_AsString. Since it is start with an underscore, it is not a public API and not documented. But I wonder why it is not documented, as it's really a useful API. PyUnicode manages a PyBytes object as the cached encoded string, and _PyUnicode_AsString will return the lvalue inside the encoded string. So we don't need anything for lifetime management in our code, this API will keep the same semantic as PyString_AsString. Now most of the test cases are passed. Just some corner things remaining, and it seems there is a 2to3 bug caused some doctests syntax not fixed. I'm going to take care of these. Regards, Haoyu Bai From seefeld at sympatico.ca Sat Jun 6 17:54:53 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Sat, 06 Jun 2009 11:54:53 -0400 Subject: [C++-sig] Py3k unicode to const char* conversion solved (Re: boost::python for Python 3.0) In-Reply-To: <1d7983e80906060823h42d7aeeet443e651f320279a2@mail.gmail.com> References: <1d7983e80906060823h42d7aeeet443e651f320279a2@mail.gmail.com> Message-ID: <4A2A914D.8090408@sympatico.ca> Haoyu Bai wrote: > Hi, > > After attempted all the hacks come from my mind, I solved the unicode > string to const char* conversion problem by simply calling a Python > C-API: _PyUnicode_AsString. Since it is start with an underscore, it > is not a public API and not documented. But I wonder why it is not > documented, as it's really a useful API. PyUnicode manages a PyBytes > object as the cached encoded string, and _PyUnicode_AsString will > return the lvalue inside the encoded string. So we don't need anything > for lifetime management in our code, this API will keep the same > semantic as PyString_AsString. > That sounds good. I would suggest to get in touch with python-dev people to ask about it (or simply submit an issue at bugs.python.org. They are pretty responsive). > Now most of the test cases are passed. Just some corner things > remaining, and it seems there is a 2to3 bug caused some doctests > syntax not fixed. I'm going to take care of these. > Excellent. It seems you are making great progress ! Stefan -- ...ich hab' noch einen Koffer in Berlin... From camejia at raytheon.com Sat Jun 6 18:45:04 2009 From: camejia at raytheon.com (Christopher A Mejia) Date: Sat, 6 Jun 2009 09:45:04 -0700 Subject: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? Message-ID: <007d01c9e6c6$248e2be0$5f8a1093@dhcp.ess.us.ray.com> Hi, I'm using pyplusplus and Boost.Python to wrap a class where one of the member functions has many (28) arguments, most of them with default values. The wrapping code handles the default arguments with the Py++ "do nothing" approach, i.e. not by using the BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS macro. I am using bjam to build my project, and am confident that the code produced by Py++ should compile, because the other member functions with similar (but fewer) arguments and default arguments compile fine. I did have to increase BOOST_PYTHON_MAX_ARITY, and I am sure it is large enough to cover my arguments. I suspect I am running into a compiler limitation because I get a slightly different error message when I remove the -ftemplate-depth-128 flag (the error message appears to be truncated when I have the -ftemplate-depth-18 flag). Here is the essence of the error message, without the -ftemplate-depth-128 flag: ------------------- /data/cmejia/install/boost_1_39_0/boost/python/detail/invoke.hpp: In function `PyObject* boost::python::detail::invoke(... /data/cmejia/install/boost_1_39_0/boost/python/detail/caller.hpp:223: instantiated from `PyObject* ... /data/cmejia/install/boost_1_39_0/boost/python/object/py_function.hpp:38: instantiated from `PyObject* ... dcs.cpp:9: instantiated from here /data/cmejia/install/boost_1_39_0/boost/python/detail/invoke.hpp:75: error: must use .* or ->* to call pointer-to-member function in `f (...)' ------------------- Which doesn't make sense because none of the other code has needed to use .* or ->*. The last 19 arguments to this member function are all doubles, all with default values. If I remove the last 3 doubles from the wrapping code produced by pyplusplus, *and* remove them from the header file I am trying to wrap, I am able to compile successfully. (If I remove them only from my wrapping code, but not from the header file I am trying to wrap, I am unable to compile.) Anyway, this seems to indicate that the compiler is running up against some limit, and I was wondering if there was a flag to set to get past this problem. The current set of compiler flags, from bjam, are "g++" -ftemplate-depth-128 -O0 -fno-inline -Wall -g -fPIC -DBOOST_PYTHON_MAX_ARITY=40 I am using g++ version 3.4.6 on Red Hat Enterprise Linux WS release 4 (Nahant Update 6). Any information would be appreciated. Thanks in advance, --Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: From seefeld at sympatico.ca Sat Jun 6 19:15:11 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Sat, 06 Jun 2009 13:15:11 -0400 Subject: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? In-Reply-To: <007d01c9e6c6$248e2be0$5f8a1093@dhcp.ess.us.ray.com> References: <007d01c9e6c6$248e2be0$5f8a1093@dhcp.ess.us.ray.com> Message-ID: <4A2AA41F.7050206@sympatico.ca> What you describe sounds like GCC is getting confused. It may have run into wrong code, but the error message may be caused by a wrong interpretation of what you wanted to do. The fact that things work if you reduce the number of arguments, and the fact that you had to increase BOOST_PYTHON_MAX_ARITY suggests that perhaps some of the code inside boost.python does not support the arity of your call (and isn't configurable via BOOST_PYTHON_MAX_ARITY, so the compiler couldn't find a matching function for your call. Then, instead of telling you that it couldn't find a match, it thought you may have missed qualification (etc.). However, to substantiate this speculation, I'd need to see the code at the call site. But the source is so noisy due to all the macros that it's impossible to see. Could you send a preprocessed source of the relevant bits ? Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin... From roman.yakovenko at gmail.com Sun Jun 7 08:26:00 2009 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Sun, 7 Jun 2009 09:26:00 +0300 Subject: [C++-sig] const shared_ptr and py++ / boost.python In-Reply-To: References: Message-ID: <7465b6170906062326u358934c9lb740074a3255fff9@mail.gmail.com> On Thu, Jun 4, 2009 at 11:37 PM, Nicolas Regnault wrote: > Dear all: > > I am wrapping a library which relies heavily on boost smart pointers (mainly > boost::shared_ptr< T >). In particular, many functions and methods have > signatures like: > > ? void my_func( const boost::shared_ptr< MyClass > & ); > > Everything works well from within C++ programs. But I encounter a serious > problem with the wrappers I have generated with py++. Indeed, when passing > from python, something which wraps a "shared_ptr< MyClass >" to > my_function(), the code within the function apparently gets a shared_ptr > with a use_count equal to 1 (pointing to the same address), whatever the > use_count of the shared_ptr passed to the function. > > I haven't been able to see what boost.python is doing in between, but > something gets altered. Any idea of what is wrong with the wrappers ? I am pretty sure, there is nothing wrong with the generated code. It is pretty close to the code, you would write. I think it happens, because Boost.Python is able to construct smart pointer class instance( shared_ptr ) from an Python object( which holds C++ class instance ) on the fly. It does so with custom deleter. If your projects works a lot with boost::shared_ptr, may be it makes sense for you to redefine HeldType ( http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/class.html#HeldType ). -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From gjcarneiro at gmail.com Sun Jun 7 12:52:30 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 7 Jun 2009 11:52:30 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> <903DA42C-6F3C-4C6F-AAA6-146627934C60@llnl.gov> Message-ID: 2009/6/7 Taner Yildirim > Dear Mike, > > > > I noticed that the special function __call__ works in pybindgen. When I use > the custom_name as __call__, then I can get the vector as (i)! > > In cppclass.py, the program checks if the name== __call__ to treat this > case specially and then converts it into slots! I suppose, you can do the > similar thing for the tp_len, tp_iter and tp_iternext! Then, these functions > can be controlled by the user easily. I think this is also standard in > python!! > > > > For example, if I create a simple class: > > > > class b: > > def __getitem__(self,i): > > return i > > def __len__(self): > > return 100 > > > > then in python, you can do the iteration and len as follow: > > > > a=b() > > for i in a: print i > > len(a) > > > > In short, I think pybindgen should also allow user to overload these > special functions easily for class. Since it is already done for __call__, > why not do it for others, in particular for tp_iter and tp_iternext. Then > the user can create his/her own class with custom special functions easily! > > > > Gustavo: Do you think this is doable easily in pybindgen??? My > understanding is that the only thing that I need is to replace the ?NONE? > with my function name for the given tp_slot in the generated wrapper code!! > Is this correct or there is more to it!!! > It has to be checked case-by-case. tp_iter maybe can be supported generically, however tp_iternext is more complicated, and I don't think you can assign an arbitrary C++ method/function to it. Especially if you wan the generated code to be efficient. It needs to check the C++ iterator, see if it has reached container.end(), and raise StopIteration if so. There is no generic C++ code that can do it. Python and C++ are just too different in this respect. tp_call was easy because the slot expects a C function that is the same thing as a normal python method wrapper. But not all slots expect this signature, some are very specific. > Back to my toy std:vector class-wrapper that I send in the previous > email, I found out why the access with the pointer is too slow! > > I had defined a get_function: > > > > int get(VecI vec, Iter_VecI it) { return *it; } > > > > which allows me to get the value of container using the iterator. However > in this function, the first parameter is the huge-array! Hence to get a > value, I was passing the whole array into the function!! I had done this in > this way, because the add_function_as_method expects the first parameter to > be the class type (i.e. VecI). > In tests/foomodulegen.py: ## add a function that appears as a method of an object SomeObject.add_function_as_method('some_object_get_something_prefixed', ReturnValue.new('std::string'), [Parameter.new('const SomeObject*', 'obj', transfer_ownership=False), Parameter.new('std::string', 'something')], custom_name='get_something_prefixed') This means that the function/method will take a pointer as first parameter, not a value, thus avoiding the value copy performance penalty. Maybe this is what you want? If you do not like pointers you can also try references. > Since in c++, we get the value of container by simply *iterator, I thought > we should do the same thing here. > > Therefore, I now modified the get_function as > > > > int get (Iter_VecI it) {return *it;} > > > > and put this function to iter_class as method with custom name ?val? and > also with ?__call__?. > > > > Hence now I can iterate over the vector-items as > > Iter=vi.begin() > > While (iter != vi.end()): > > Item_value = it() # thanks to to __call__ slot! > > Item_value = it.val() # maybe this is better notation!! > > > > Anyway, now the sum_test in the previous email takes about the same time as > the indexing! > > > > I am still not able to add_function_as_constructor (for my FromList > function)? Unfortunatley, pybindgen expects pointer-function for this. Then > I do not know how to construct my vector without returning it!!! Any > suggestion would be appreciated very much! > add_function_as_constructor expects a function that returns the object. Usually the return value is like ReturnValue("MyObject*", caller_owns_return=True), so that the object is constructed only once and owership is passed to the python wrapper. Maybe I do not quite understand you problem. > I do not see why the add_function_as_method has a different convention than > as add_function_as_contructor??? In any case, from above get_function > example, I learned that add_function_as_method is not a good thing if the > first-parameter requires a huge data passing to the function!! Probably I am > using add_function_as_method incorrectly! I?ll have to look at more in > foomodegenerate.py which is my main source for learning the pybindgen. As a > fortran user over 20 years, this c++ pointers, references, etc are all very > confusion to me!!! > > > A final question: Can?t we make the container-class inherited from > cppclass???? If this is easy to do, then we can add all the methods that > std:container support easily as done in my toy std-int class wrapper! > I am beginning to think this is a good idea. I added a bug report for this, though I can give no guarantees when I will handle it: https://bugs.launchpad.net/pybindgen/+bug/384488 Regards, -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From gjcarneiro at gmail.com Sun Jun 7 20:27:09 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 7 Jun 2009 19:27:09 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: <363F8F11588E41A884BAC48D15391609@ncnrwin.ncnr.nist.gov> References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> <903DA42C-6F3C-4C6F-AAA6-146627934C60@llnl.gov> <363F8F11588E41A884BAC48D15391609@ncnrwin.ncnr.nist.gov> Message-ID: 2009/6/7 Taner Yildirim > Dear Gustavo, > > > > Thanks for your detailed response. I hope pybindgen soon will have the full > support for the stl-containers! > > I am looking forward to that particular version! Hopefully in the near > future, I will learn more about pybindgen and maybe contribute something to > the development of the project. It?s a great tool which has a much better > and clean approach than swig and boos.python in my view!!! > > > > Back to my toy VecI example, unfortunately I am still having problem in > getting the ?add_function_as_contructor? work. > > Here is exactly what I am trying to do. In my earlier VecI class example, I > had this function in tst1.h: > > > > void build_from_list( VecI &new_vec, std::vector vi) { > > new_vec.clear(); new_vec=VecI(vi);} > > > > In c++ side, VecI is just a typedef for std::vector. Hence this > function works. For python side, pybindgen thinks that pybindgen container > (i.e. std::Vector) and my VecI class are two different things. Hence > this function works as a bridge between them. > > Anyway, when I add this function as method with custom_name=FromList, I was > able to build my vector from the python list: > > > > V=VecI() > > V=V.FromList(range(10)) > > > > Now, I want to add this function as contructor rather than method so that I > can simple say myvector=VecI(range(10)), let say! > > In, pybindgen CppFunctionAsConstructor, it says that the first parameter > should return ?a pointer to the class? (not the class itself!). > > Hence, I changed my code like this: > > > > VecI *from_list( std::vector vi){ > > VecI * p_new_vec=&vi; > > return p_new_vec;} > > > > Then, I add this to pybindgen as > > > > > veci.add_function_as_constructor('from_list',ReturnValue.new('VecI*',caller_owns_return=True), > > [Parameter.new('std::vector',' vec_int')]) > > > > Of course, technically, everything seemed work! But the problem is that > when I construct my vector, I am getting pointer and not the vector itself! > > What I mean is the following: > > > > V=VecI(range(10)) # works, python does not complain it! > > > > But then V.size() returns a huge integer. I think V is now a pointer and > not the vector. > > Probably the fix is very simple but I just do not see it! I tried to use & > instead of **, * > > but then compiler complain that it can not convert int to int*. > > > > Hope my problem is clear now. > Well, it is clear to me that you do not have deep knowledge of C/C++ memory management, no offense intended :-) Let's look at your code: VecI *from_list( std::vector vi){ VecI * p_new_vec=&vi; return p_new_vec; } The variable p_new_vec will be a pointer that points to the 'vi' variable. 'vi' is a parameter passed by value, which means it is an "automatic variable". When the function exits, 'vi' goes out of scope and is deleted. However, you will have returned a pointer to what used to be a variable but does not exist anymore; it points to garbage now. The simplest solution is to copy the passed-in value, using a copy constructor. I would do a code like this insterad: VecI *from_list( std::vector const &vi){ VecI * p_new_vec= new std::vector (vi); return p_new_vec; } This should work better. Good luck. -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From gjcarneiro at gmail.com Mon Jun 8 01:24:20 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Mon, 8 Jun 2009 00:24:20 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: <2BC952EE4CF7460CB6216483588D0DD3@ncnrwin.ncnr.nist.gov> References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> <903DA42C-6F3C-4C6F-AAA6-146627934C60@llnl.gov> <363F8F11588E41A884BAC48D15391609@ncnrwin.ncnr.nist.gov> <2BC952EE4CF7460CB6216483588D0DD3@ncnrwin.ncnr.nist.gov> Message-ID: 2009/6/7 Taner Yildirim > Dear Gustavo, > > > > Thanks for the detailed explanation, which fixed the problem!. As I > indicated in my previous email, I am mainly a Fortran programmer and do not > have much experience with c++. I used to wrap my programs with F2Py but then > decided to do right thing and learn c/c++ and rewrite some of the > applications. Thanks to your pybindgen code, the wrapping them for python > would be the easiest part!! > > > > Back to pybindgen containers, I noticed a strange behavior. I did not check > the bug reports. Sorry if this was already reported. > > I have the following code: > > > > mod=Module('tst1') > > mod.add_include('"tst1.h"') > > mod.add_container('std::vector', 'int', > 'vector',custom_name="std_vec_int") > > > > In python, I do this: > > > > A=std_vec_int([1,2,3]) > > B=std_vec_int([1,2,3]) > > > > But then > > A==B returns false??? > > > > Similarly, I do c=std_vec_int(a) and then check a==c which returns false!!! > Type comparison, like many other things, is not yet implemented in pybindgen. Like I said, you can do a lot of things by simply "casting" in python to a list, i.e. instead of: A=std_vec_int([1,2,3]) do: A=list(std_vec_int([1,2,3])) It's an ugly workaround but it works ok (although the performance isn't great). > > > If I do list(A) and list(B), I get the right list. It looks like ?==? not > working or I am again missing something very trivial! > You are not missing anything, it's just not implemented. Unfortunately Python 2.x has some default comparison function; this is fixed in Python 3.x, which just raises an exception if the comparison operation is not implemented. > By the way, this observation is based on pybindgen-0.10.0 and mingw (gcc > 4.4.0). > > > > Best regards > > Taner > > > > > ******************************************************************************** > Dr. Taner Yildirim, Ph.D. Physicist, > Adjunct Associate Professor, > Materials Science and Engineering, > University of Pennsylvania > and > Computational and Neutron Science Team Leader, > NIST Center for Neutron Research, > National Institute of Standards and Technology, > 100 Bureau Drive, Gaithersburg, MD 20899-6100. > Email: taner at nist.gov or taner at seas.upenn.edu > PHONE: 301-975-6228 FAX : 301-921-9847 > Web : http://webster.ncnr.nist.gov/staff/taner > > ******************************************************************************** > > > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From wjoergensen at yahoo.com Mon Jun 8 12:28:28 2009 From: wjoergensen at yahoo.com (Werner Joergensen) Date: Mon, 8 Jun 2009 03:28:28 -0700 (PDT) Subject: [C++-sig] [Py++] registration order problem in constructor Message-ID: <411326.12970.qm@web111812.mail.gq1.yahoo.com> Hello, the Py++ documentation has several recommendations how to solve registration order problems with member functions. Like for example this solution (from the documentation): >s2 = mb.class_( "S2" ) >s2.member_function( "do_smth" ).use_overload_macro = True But what to do if the class constructors have a registration order problem? I tried >test = mb.class_( "Test" ) >for c in test.constructors(): > c.use_overload_macro = True but this doesn't solve the problem described below. The (condensed) code I want to pythonify is: -------------------- #include #include class Test:public std::string{ public: Test(std::string); Test(char*); Test(char); void printme(); }; Test::Test(std::string s):std::string(s){ } Test::Test(char *s):std::string(s){ } Test::Test(char c):std::string(1,c){ } void Test::printme(){ std::cout<<*this<In [1]: import testmodule >In [2]: t=testmodule.Test("Hello") >In [3]: t.printme() >H #(expected "Hello" here) This indicated that the Test(char) constructor is applied although a string was given. How can I solve the problem? If possible only with module builder functions, without changes to the C code? Thanks Werner From camejia at raytheon.com Mon Jun 8 15:59:45 2009 From: camejia at raytheon.com (Christopher A Mejia) Date: Mon, 8 Jun 2009 06:59:45 -0700 Subject: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? References: <007d01c9e6c6$248e2be0$5f8a1093@dhcp.ess.us.ray.com> <4A2AA41F.7050206@sympatico.ca> Message-ID: <006f01c9e841$6153f420$5f8a1093@dhcp.ess.us.ray.com> Stefan, Thanks for your response. The API that I am trying to wrap is the Data Client Server portion of the U.S. Naval Research Labs' SIMDIS Analysis & Display Tool (simdis.nrl.navy.mil). This is free software, but does require registration and there is a human in the loop of the approval process. Anyway, I don't feel comfortable redistributing the software (or preprocessed details), but if you're really interested you could obtain it directly yourself. Apparently this API is a good test case for pushing some of the limits of Boost.Python. Since you agree that this is probably a compiler issue, it's unlikely that you or anyone else would be able to reproduce the same errors anyway. I came up with a workaround, which is to wrap a version of the function without the last three arguments, i.e. from Python I am OK with always having those last three parameters set to their default values. But if anyone has any other ideas to try, I'd like to hear them. Thanks again for your time, --Chris ----- Original Message ----- From: Stefan Seefeld To: Development of Python/C++ integration Sent: Saturday, June 06, 2009 10:15 AM Subject: Re: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? What you describe sounds like GCC is getting confused. It may have run into wrong code, but the error message may be caused by a wrong interpretation of what you wanted to do. The fact that things work if you reduce the number of arguments, and the fact that you had to increase BOOST_PYTHON_MAX_ARITY suggests that perhaps some of the code inside boost.python does not support the arity of your call (and isn't configurable via BOOST_PYTHON_MAX_ARITY, so the compiler couldn't find a matching function for your call. Then, instead of telling you that it couldn't find a match, it thought you may have missed qualification (etc.). However, to substantiate this speculation, I'd need to see the code at the call site. But the source is so noisy due to all the macros that it's impossible to see. Could you send a preprocessed source of the relevant bits ? Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin... _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org http://mail.python.org/mailman/listinfo/cplusplus-sig The following line is added for your protection and will be used for analysis if this message is reported as spam: (Raytheon Analysis: IP=82.94.164.166; e-from=cplusplus-sig-bounces+camejia=raytheon.com at python.org; from=seefeld at sympatico.ca; date=Jun 6, 2009 5:15:33 PM; subject=Re: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ?) From seefeld at sympatico.ca Mon Jun 8 16:39:11 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Mon, 08 Jun 2009 10:39:11 -0400 Subject: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? In-Reply-To: <006f01c9e841$6153f420$5f8a1093@dhcp.ess.us.ray.com> References: <007d01c9e6c6$248e2be0$5f8a1093@dhcp.ess.us.ray.com> <4A2AA41F.7050206@sympatico.ca> <006f01c9e841$6153f420$5f8a1093@dhcp.ess.us.ray.com> Message-ID: <4A2D228F.8020406@sympatico.ca> Christopher A Mejia wrote: > Stefan, > > Thanks for your response. The API that I am trying to wrap is the > Data Client Server portion of the U.S. Naval Research Labs' SIMDIS > Analysis & Display Tool (simdis.nrl.navy.mil). This is free software, > but does require registration and there is a human in the loop of the > approval process. Anyway, I don't feel comfortable redistributing the > software (or preprocessed details), but if you're really interested > you could obtain it directly yourself. Sorry, I'm not interested enough to register at this time. :-) I merely tried to be helpful in debugging the issue. > Apparently this API is a good test case for pushing some of the limits > of Boost.Python. Since you agree that this is probably a compiler > issue, it's unlikely that you or anyone else would be able to > reproduce the same errors anyway. The only compiler issue I mentioned was a misleading error message. I still believe the real issue might be inside boost.python (where some code may not support your high arity, and can't be configured with BOOST_PYTHON_MAX_ARITY), but I lack details to be able to confirm this. > I came up with a workaround, which is to wrap a version of the > function without the last three arguments, i.e. from Python I am OK > with always having those last three parameters set to their default > values. But if anyone has any other ideas to try, I'd like to hear them. It would really be best to construct the proverbial "minimal complete test case" that allows others to reproduce the issue. You may consider writing some stand-alone test using similar calls as in your code, which you then may freely send do this list, or file as an issue to the boost tracker. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From sipickles at googlemail.com Mon Jun 8 21:31:23 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Mon, 08 Jun 2009 20:31:23 +0100 Subject: [C++-sig] Getting a c++ pointer from a python object Message-ID: <4A2D670B.6000603@googlemail.com> Hi, I am trying to get a c++ pointer to a python object. I am using DirectPython, and a function in that library, getDevice(), "returns the address of the object" - a IDirect3DDevice9*, already created using DirectPython code. Using boost::python, I have an extension module function with this signature: // c++ void TestClass::Init( IDirect3DDevice9* device ); I tried to call this function with: #python, called after device is created: from testClass import TestClass test = TestClass() test.Init( d3d.getDevice() ) Here's the error I get: ################## Traceback (most recent call last): File "client.py", line 31, in __init__ test.Init(d3d.getDevice()) Boost.Python.ArgumentError: Python argument types in TestClass.Init(TestClass, long) did not match C++ signature: Init(class TestClass {lvalue}, struct IDirect3DDevice9 *) ################## But got a type mismatch. DirectPython gives the address as a long, when c++ wants a pointer to a struct. I then tried this with the same python calling code: void TestClass::Init(void* deviceAddress) { IDirect3DDevice9* m_device = static_cast(deviceAddress); } Of course I get a similar error: ################## Traceback (most recent call last): File "client.py", line 33, in __init__ test.Init(d3d.getDevice()) Boost.Python.ArgumentError: Python argument types in TestClass.Init(TestClass, long) did not match C++ signature: Init(class TestClass {lvalue}, void *) ################## How can I get the address from python to c++? Thanks Simon From roman.yakovenko at gmail.com Tue Jun 9 07:22:30 2009 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Tue, 9 Jun 2009 08:22:30 +0300 Subject: [C++-sig] [Py++] registration order problem in constructor In-Reply-To: <411326.12970.qm@web111812.mail.gq1.yahoo.com> References: <411326.12970.qm@web111812.mail.gq1.yahoo.com> Message-ID: <7465b6170906082222n2b7fdcdah72276522d72e83c3@mail.gmail.com> On Mon, Jun 8, 2009 at 1:28 PM, Werner Joergensen wrote: > > Hello, > the Py++ documentation has several recommendations how to solve registration order problems with member functions. > Like for example this solution (from the documentation): >>s2 = mb.class_( "S2" ) >>s2.member_function( "do_smth" ).use_overload_macro = True > > But what to do if the class constructors have a registration order problem? I tried >>test = mb.class_( "Test" ) >>for c in test.constructors(): >> ?c.use_overload_macro = True > but this doesn't solve the problem described below. > > The (condensed) code I want to pythonify is: > -------------------- > #include > #include > class Test:public std::string{ > public: > ?Test(std::string); > ?Test(char*); > ?Test(char); > ?void printme(); > }; > Test::Test(std::string s):std::string(s){ > } > Test::Test(char *s):std::string(s){ > } > Test::Test(char c):std::string(1,c){ > } > void Test::printme(){ > ?std::cout<<*this< } > -------------------- > > However, a test of the generated python module shows: >>In [1]: import testmodule >>In [2]: t=testmodule.Test("Hello") >>In [3]: t.printme() >>H ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?#(expected "Hello" here) > > This indicated that the Test(char) constructor is applied although a string was given. > How can I solve the problem? Py++ doesn't help you here. > If possible only with module builder functions, without changes to the C code? Partially. I suggest you to add few free functions and expose them: Test* create_test_from_char( char ){...} Test* create_test_from_string( std::string ){...} Test* create_test_from_char_array( const char* ){...} Thus you will not have to change your original code and the user code will be more "readable". There are few other alternatives, but I think you've got the idea. -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From wjoergensen at yahoo.com Tue Jun 9 20:22:45 2009 From: wjoergensen at yahoo.com (Werner Joergensen) Date: Tue, 9 Jun 2009 11:22:45 -0700 (PDT) Subject: [C++-sig] [Py++] registration order problem in constructor In-Reply-To: <7465b6170906082222n2b7fdcdah72276522d72e83c3@mail.gmail.com> References: <411326.12970.qm@web111812.mail.gq1.yahoo.com> <7465b6170906082222n2b7fdcdah72276522d72e83c3@mail.gmail.com> Message-ID: <308813.32668.qm@web111804.mail.gq1.yahoo.com> Dear Roman, thank you very much for the suggested workaround. Could you answer the additional questions below? ... > > This indicated that the Test(char) constructor is applied although a string > was given. > > How can I solve the problem? > > Py++ doesn't help you here. > > > If possible only with module builder functions, without changes to the C code? > > Partially. I suggest you to add few free functions and expose them: > > Test* create_test_from_char( char ){...} > Test* create_test_from_string( std::string ){...} > Test* create_test_from_char_array( const char* ){...} > > Thus you will not have to change your original code and the user code > will be more "readable". > There are few other alternatives, but I think you've got the idea. Yes, with your suggestion I can at least work with the code. But would you mind mentioning the other alternatives? My aim is to construct the class like the "t=testmodule.Test("Hello")" above. I was not able to use the suggested "create_*" functions as fake constructors "test.add_fake_constructors( mb.calldefs( 'create_test_from_char' ) )". Do they have to return an auto_ptr for this purpose? Thank you Werner From roman.yakovenko at gmail.com Tue Jun 9 21:15:51 2009 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Tue, 9 Jun 2009 22:15:51 +0300 Subject: [C++-sig] [Py++] registration order problem in constructor In-Reply-To: <308813.32668.qm@web111804.mail.gq1.yahoo.com> References: <411326.12970.qm@web111812.mail.gq1.yahoo.com> <7465b6170906082222n2b7fdcdah72276522d72e83c3@mail.gmail.com> <308813.32668.qm@web111804.mail.gq1.yahoo.com> Message-ID: <7465b6170906091215y32baf889wae6cc8fa64d1409@mail.gmail.com> On Tue, Jun 9, 2009 at 9:22 PM, Werner Joergensen wrote: > > Dear Roman, > thank you very much for the suggested workaround. Could you answer the additional questions below? > > ... >> > This indicated that the Test(char) constructor is applied although a string > >> was given. >> > How can I solve the problem? >> >> Py++ doesn't help you here. >> >> > If possible only with module builder functions, without changes to the C code? >> >> Partially. I suggest you to add few free functions and expose them: >> >> Test* create_test_from_char( char ){...} >> Test* create_test_from_string( std::string ){...} >> Test* create_test_from_char_array( const char* ){...} >> >> Thus you will not have to change your original code and the user code >> will be more "readable". >> There are few other alternatives, but I think you've got the idea. > > Yes, with your suggestion I can at least work with the code. > > But would you mind mentioning the other alternatives? My aim is to construct the class like the "t=testmodule.Test("Hello")" above. In my opinion, you are working with "tricky" overloading - "char", "char*" and "std::string". I could be wrong, but Python doesn't have type "char", but string and unicode. So may be you should not expose "Test(char)" and "Test(char*)" constructors at all. Anyway, "pure/poor" Python user will/should not understand the difference between those constructors. Another option, could be to create a Python class which derives from the Test class ( you can rename it ) and implement dispatch logic by yourself. You also can create fake constructor in C++, which takes boost::python::object and makes the desired dispatching. > I was not able to use the suggested "create_*" functions as fake constructors "test.add_fake_constructors( mb.calldefs( 'create_test_from_char' ) )". Do they have to return an auto_ptr for this purpose? Yes. It was my mistake, sorry. HTH -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From wjoergensen at yahoo.com Tue Jun 9 22:43:53 2009 From: wjoergensen at yahoo.com (Werner Joergensen) Date: Tue, 9 Jun 2009 13:43:53 -0700 (PDT) Subject: [C++-sig] [Py++] registration order problem in constructor In-Reply-To: <7465b6170906091215y32baf889wae6cc8fa64d1409@mail.gmail.com> References: <411326.12970.qm@web111812.mail.gq1.yahoo.com> <7465b6170906082222n2b7fdcdah72276522d72e83c3@mail.gmail.com> <308813.32668.qm@web111804.mail.gq1.yahoo.com> <7465b6170906091215y32baf889wae6cc8fa64d1409@mail.gmail.com> Message-ID: <375482.5773.qm@web111815.mail.gq1.yahoo.com> Thanks! I can't express my gratitude for your help on this list. I think my questions are answered. I'll see if I can implement your solutions. Werner > > But would you mind mentioning the other alternatives? My aim is to construct > the class like the "t=testmodule.Test("Hello")" above. > > In my opinion, you are working with "tricky" overloading - "char", > "char*" and "std::string". I could be wrong, but Python doesn't have > type "char", but string and unicode. So may be you should not expose > "Test(char)" and "Test(char*)" constructors at all. Anyway, > "pure/poor" Python user will/should not understand the difference > between those constructors. > > Another option, could be to create a Python class which derives from > the Test class ( you can rename it ) and implement dispatch logic by > yourself. You also can create fake constructor in C++, which takes > boost::python::object and makes the desired dispatching. > > > I was not able to use the suggested "create_*" functions as fake constructors > "test.add_fake_constructors( mb.calldefs( 'create_test_from_char' ) )". Do they > have to return an auto_ptrfor this purpose? > > Yes. It was my mistake, sorry. > > HTH > > -- > Roman Yakovenko > C++ Python language binding > http://www.language-binding.net/ > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig From taner at seas.upenn.edu Wed Jun 10 07:49:08 2009 From: taner at seas.upenn.edu (Taner Yildirim) Date: Wed, 10 Jun 2009 01:49:08 -0400 Subject: [C++-sig] a new version of pybindgen with full STL containers method support!!! In-Reply-To: References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> <903DA42C-6F3C-4C6F-AAA6-146627934C60@llnl.gov> <363F8F11588E41A884BAC48D15391609@ncnrwin.ncnr.nist.gov> <2BC952EE4CF7460CB6216483588D0DD3@ncnrwin.ncnr.nist.gov> Message-ID: <1FED9F02AF0147AEBFFC49F736436C06@ncnrwin.ncnr.nist.gov> Dear Gustavo and Mike, When you have a chance, please have a look at the website: http://www.ncnr.nist.gov/staff/taner/pybindgen/doc_installation_features.htm l Implementing my toy std::vector class to a generic type in pybindgen code turned out to be quite easy! All I had to do is to write a function in module.py which basically does the same thing as add_container but instead returning a container, it also create a std::vector class, add the methods to it and then return the class back to user. In this way, without messing up with the container module or any other pybindgen modules, I was able to combine the container and the stl:vector class with an arbitrary user-defined type!!! Since we now get a class when the container is added, the user can add additional functions as method as well. In the two examples given in the website, I demonstrated this adding sum() and operator== for a complex type for the std:vector. The usage is almost the same as before but instead using add_container, now we have to use stl_add_container. The rest is automatic. Also, thanks to __call__, we can access the vector elements as vec(index). vec() gives the iterable container. Hence list(Vec()) works for python list. Anyway, I created a brief website to explain my approach. I discuss two examples in details. The first one is regular std::vector with double and integer types. The other is std:vector of a custom type! Making this code work for other container would be trivial too. But first, I would like to get some feedback from you guys to see if you like the approach. If you are interested, you can get the modified module.py (which is the same as original but with a new function add_stl_container) at my website. Please have a look at the web site: http://www.ncnr.nist.gov/staff/taner/pybindgen/ Best regards Taner PS: I created the website very quickly using Sphinx. Sorry for the typos or strange behaviors with the web design. I'll fix it later. **************************************************************************** **** Dr. Taner Yildirim, Ph.D. Physicist, Adjunct Associate Professor, Materials Science and Engineering, University of Pennsylvania and Computational and Neutron Science Team Leader, NIST Center for Neutron Research, National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, MD 20899-6100. Email: taner at nist.gov or taner at seas.upenn.edu PHONE: 301-975-6228 FAX : 301-921-9847 Web : http://webster.ncnr.nist.gov/staff/taner **************************************************************************** **** _____ From: Gustavo Carneiro [mailto:gjcarneiro at gmail.com] Sent: Sunday, June 07, 2009 7:24 PM To: taner at seas.upenn.edu Cc: J. Michael Owen; cplusplus-sig at python.org Subject: Re: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? 2009/6/7 Taner Yildirim Dear Gustavo, Thanks for the detailed explanation, which fixed the problem!. As I indicated in my previous email, I am mainly a Fortran programmer and do not have much experience with c++. I used to wrap my programs with F2Py but then decided to do right thing and learn c/c++ and rewrite some of the applications. Thanks to your pybindgen code, the wrapping them for python would be the easiest part!! Back to pybindgen containers, I noticed a strange behavior. I did not check the bug reports. Sorry if this was already reported. I have the following code: mod=Module('tst1') mod.add_include('"tst1.h"') mod.add_container('std::vector', 'int', 'vector',custom_name="std_vec_int") In python, I do this: A=std_vec_int([1,2,3]) B=std_vec_int([1,2,3]) But then A==B returns false??? Similarly, I do c=std_vec_int(a) and then check a==c which returns false!!! Type comparison, like many other things, is not yet implemented in pybindgen. Like I said, you can do a lot of things by simply "casting" in python to a list, i.e. instead of: A=std_vec_int([1,2,3]) do: A=list(std_vec_int([1,2,3])) It's an ugly workaround but it works ok (although the performance isn't great). If I do list(A) and list(B), I get the right list. It looks like "==" not working or I am again missing something very trivial! You are not missing anything, it's just not implemented. Unfortunately Python 2.x has some default comparison function; this is fixed in Python 3.x, which just raises an exception if the comparison operation is not implemented. By the way, this observation is based on pybindgen-0.10.0 and mingw (gcc 4.4.0). Best regards Taner **************************************************************************** **** Dr. Taner Yildirim, Ph.D. Physicist, Adjunct Associate Professor, Materials Science and Engineering, University of Pennsylvania and Computational and Neutron Science Team Leader, NIST Center for Neutron Research, National Institute of Standards and Technology, 100 Bureau Drive, Gaithersburg, MD 20899-6100. Email: taner at nist.gov or taner at seas.upenn.edu PHONE: 301-975-6228 FAX : 301-921-9847 Web : http://webster.ncnr.nist.gov/staff/taner **************************************************************************** **** -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From camejia at raytheon.com Fri Jun 12 02:13:10 2009 From: camejia at raytheon.com (Christopher A Mejia) Date: Thu, 11 Jun 2009 17:13:10 -0700 Subject: [C++-sig] Fw: g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? Message-ID: <003a01c9eaf2$91a27c60$5f8a1093@dhcp.ess.us.ray.com> Stefan / cplusplus-sig, Based on Stefan's encouragement, I developed a minimal complete test case exhibiting this problem, contained in the attached files. The compilation command I used was: g++ -DBOOST_PYTHON_MAX_ARITY=40 -I/data/cmejia/install/boost_1_39_0 -I/data/include/python2.6 -c -o foo.o foo.cpp As I explained earlier in the first message of this thread, this fails to compile, but if I remove one double argument towards the end (i.e. edit out one line of foo.hpp and two lines of foo.cpp) it does compile successfully. I would appreciate any further insight into this problem, and hopefully we can use this test case to make Boost.Python an even better tool. Thanks in advance, --Chris P.S. Unfortunately, I was unable to include preprocessed source from "g++ ... -E foo.cpp" because the result was too large for the mailing list system. Christopher A Mejia wrote: > Stefan, > > Thanks for your response. The API that I am trying to wrap is the > Data Client Server portion of the U.S. Naval Research Labs' SIMDIS > Analysis & Display Tool (simdis.nrl.navy.mil). This is free software, > but does require registration and there is a human in the loop of the > approval process. Anyway, I don't feel comfortable redistributing the > software (or preprocessed details), but if you're really interested > you could obtain it directly yourself. Sorry, I'm not interested enough to register at this time. :-) I merely tried to be helpful in debugging the issue. > Apparently this API is a good test case for pushing some of the limits > of Boost.Python. Since you agree that this is probably a compiler > issue, it's unlikely that you or anyone else would be able to > reproduce the same errors anyway. The only compiler issue I mentioned was a misleading error message. I still believe the real issue might be inside boost.python (where some code may not support your high arity, and can't be configured with BOOST_PYTHON_MAX_ARITY), but I lack details to be able to confirm this. > I came up with a workaround, which is to wrap a version of the > function without the last three arguments, i.e. from Python I am OK > with always having those last three parameters set to their default > values. But if anyone has any other ideas to try, I'd like to hear them. It would really be best to construct the proverbial "minimal complete test case" that allows others to reproduce the issue. You may consider writing some stand-alone test using similar calls as in your code, which you then may freely send do this list, or file as an issue to the boost tracker. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org http://mail.python.org/mailman/listinfo/cplusplus-sig The following line is added for your protection and will be used for analysis if this message is reported as spam: (Raytheon Analysis: IP=82.94.164.166; e-from=cplusplus-sig-bounces+camejia=raytheon.com at python.org; from=seefeld at sympatico.ca; date=Jun 8, 2009 2:39:33 PM; subject=Re: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ?) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: foo.cpp URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: foo.hpp URL: From camejia at raytheon.com Fri Jun 12 05:51:19 2009 From: camejia at raytheon.com (Christopher A Mejia) Date: Thu, 11 Jun 2009 20:51:19 -0700 Subject: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? References: <007d01c9e6c6$248e2be0$5f8a1093@dhcp.ess.us.ray.com> <4A2AA41F.7050206@sympatico.ca><006f01c9e841$6153f420$5f8a1093@dhcp.ess.us.ray.com> <4A2D228F.8020406@sympatico.ca> Message-ID: <003501c9eb11$0b6883a0$5f8a1093@dhcp.ess.us.ray.com> Stefan / cplusplus-sig, Based on Stefan's encouragement, I developed a minimal complete test case exhibiting this problem, contained in the attached files. The compilation command I used was: g++ -DBOOST_PYTHON_MAX_ARITY=40 -I/data/cmejia/install/boost_1_39_0 -I/data/include/python2.6 -c -o foo.o foo.cpp As I explained earlier in the first message of this thread, this fails to compile, but if I remove one double argument towards the end (i.e. edit out one line of foo.hpp and two lines of foo.cpp) it does compile successfully. I would appreciate any further insight into this problem, and hopefully we can use this test case to make Boost.Python an even better tool. Thanks in advance, --Chris ----- Original Message ----- From: Stefan Seefeld To: Development of Python/C++ integration Sent: Monday, June 08, 2009 7:39 AM Subject: Re: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? Christopher A Mejia wrote: > Stefan, > > Thanks for your response. The API that I am trying to wrap is the > Data Client Server portion of the U.S. Naval Research Labs' SIMDIS > Analysis & Display Tool (simdis.nrl.navy.mil). This is free software, > but does require registration and there is a human in the loop of the > approval process. Anyway, I don't feel comfortable redistributing the > software (or preprocessed details), but if you're really interested > you could obtain it directly yourself. Sorry, I'm not interested enough to register at this time. :-) I merely tried to be helpful in debugging the issue. > Apparently this API is a good test case for pushing some of the limits > of Boost.Python. Since you agree that this is probably a compiler > issue, it's unlikely that you or anyone else would be able to > reproduce the same errors anyway. The only compiler issue I mentioned was a misleading error message. I still believe the real issue might be inside boost.python (where some code may not support your high arity, and can't be configured with BOOST_PYTHON_MAX_ARITY), but I lack details to be able to confirm this. > I came up with a workaround, which is to wrap a version of the > function without the last three arguments, i.e. from Python I am OK > with always having those last three parameters set to their default > values. But if anyone has any other ideas to try, I'd like to hear them. It would really be best to construct the proverbial "minimal complete test case" that allows others to reproduce the issue. You may consider writing some stand-alone test using similar calls as in your code, which you then may freely send do this list, or file as an issue to the boost tracker. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org http://mail.python.org/mailman/listinfo/cplusplus-sig The following line is added for your protection and will be used for analysis if this message is reported as spam: (Raytheon Analysis: IP=82.94.164.166; e-from=cplusplus-sig-bounces+camejia=raytheon.com at python.org; from=seefeld at sympatico.ca; date=Jun 8, 2009 2:39:33 PM; subject=Re: [C++-sig] g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ?) -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: foo.hpp URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: foo.cpp URL: From wjoergensen at yahoo.com Sat Jun 13 10:48:42 2009 From: wjoergensen at yahoo.com (Werner Joergensen) Date: Sat, 13 Jun 2009 01:48:42 -0700 (PDT) Subject: [C++-sig] [Py++] operators() got an unexpected keyword argument 'decl_type' Message-ID: <922817.1446.qm@web111801.mail.gq1.yahoo.com> Hello, I need to exclude some operators from my wrapped python classes, but I seem not to be able to access the operators. What is wrong with this code: ----------8<---------- from pyplusplus import module_builder files=["test.H"] mb = module_builder.module_builder_t( files=files, ) print mb.operators() ----------8<---------- (For completeness, test.H contains nothing but "class X{public: int i(); };" The following error, however, occurs independent of test.H.) On the last line the script produces: ----------8<---------- Traceback (most recent call last) /home/wj/tmp/ in () /usr/lib/python2.5/site-packages/pyplusplus/module_builder/builder.py in operators(self, name, symbol, return_type, arg_types, decl_type, header_dir, header_file, recursive) 541 , header_dir=header_dir 542 , header_file=header_file --> 543 , recursive=recursive ) 544 545 def member_function( self, name=None, function=None, return_type=None, arg_types=None, header_dir=None, header_file=None, recursive=None ): : operators() got an unexpected keyword argument 'decl_type' ----------8<---------- The installation is latest gccxml from CVS, pygccxml-1.0.0 and pyplusplus-1.0.0 with python 2.5.2. Same result with python 2.6. Please help. Is the use of mb.operators() wrong, or what else might be the problem? Werner From bennie.koerner at web.de Sat Jun 13 20:09:05 2009 From: bennie.koerner at web.de (Benjamin Koerner) Date: Sat, 13 Jun 2009 20:09:05 +0200 Subject: [C++-sig] [pyplusplus] functions returning double& Message-ID: <998236788@web.de> How can one allow Py++ to handle member functions that return a reference to an immutable type? For example this X::getval function: class X{ double xval; public: double& getval(){return xval;} }; I don't care about the reference. All I need is the double value, which is only used as an r-value. Can one use a call policy in this situation, or does one have to write custom wrapper code? Regards, Bennie ____________________________________________________________ Text: GRATIS f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT! Jetzt freischalten unter http://movieflat.web.de From roman.yakovenko at gmail.com Sun Jun 14 06:06:20 2009 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Sun, 14 Jun 2009 07:06:20 +0300 Subject: [C++-sig] [Py++] operators() got an unexpected keyword argument 'decl_type' In-Reply-To: <922817.1446.qm@web111801.mail.gq1.yahoo.com> References: <922817.1446.qm@web111801.mail.gq1.yahoo.com> Message-ID: <7465b6170906132106ke4d64ceq66bbd04e0dee5cb7@mail.gmail.com> On Sat, Jun 13, 2009 at 11:48 AM, Werner Joergensen wrote: > > Hello, > I need to exclude some operators from my wrapped python classes, but I seem not to be able to access the operators. > > What is wrong with this code: > ----------8<---------- > from pyplusplus import module_builder > files=["test.H"] > mb = module_builder.module_builder_t( > ? ? ? ?files=files, > ? ?) > print mb.operators() > ----------8<---------- > (For completeness, test.H contains nothing but "class X{public: ?int i(); };" > The following error, however, occurs independent of test.H.) > > On the last line the script produces: > ----------8<---------- > ? ? ? ? ? ? Traceback (most recent call last) > /home/wj/tmp/ in () > /usr/lib/python2.5/site-packages/pyplusplus/module_builder/builder.py in operators(self, name, symbol, return_type, arg_types, decl_type, header_dir, header_file, recursive) > ? ?541 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?, header_dir=header_dir > ? ?542 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?, header_file=header_file > --> 543 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?, recursive=recursive ) > ? ?544 > ? ?545 ? ? def member_function( self, name=None, function=None, return_type=None, arg_types=None, header_dir=None, header_file=None, recursive=None ): > : operators() got an unexpected keyword argument 'decl_type' > ----------8<---------- > > The installation is latest gccxml from CVS, pygccxml-1.0.0 and pyplusplus-1.0.0 with python 2.5.2. Same result with python 2.6. > > Please help. Is the use of mb.operators() wrong, or what else might be the problem? The version you use contains such bug. It is fixed in SVN. If you don't want to upgrade you can use the following code: mb.global_ns.operators() # query global namespace for all operators HTH -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From roman.yakovenko at gmail.com Sun Jun 14 06:19:33 2009 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Sun, 14 Jun 2009 07:19:33 +0300 Subject: [C++-sig] [pyplusplus] functions returning double& In-Reply-To: <998236788@web.de> References: <998236788@web.de> Message-ID: <7465b6170906132119u1871f447qf6df928c533f2d06@mail.gmail.com> On Sat, Jun 13, 2009 at 9:09 PM, Benjamin Koerner wrote: > How can one allow Py++ to handle member functions that return a reference to an immutable type? > > For example this X::getval function: > class X{ > double xval; > public: > ?double& getval(){return xval;} > }; > > I don't care about the reference. All I need is the double value, which is only used as an r-value. Can one use a call policy in this situation, or does one have to write custom wrapper code? Did you read Boost.Python manuals? http://www.boost.org/doc/libs/1_39_0/libs/python/doc/v2/copy_non_const_reference.html from pyplusplus.module_builder import call_policies ... mb.calldef( '::X::getval' ).call_policies = call_policies.return_value_policy( call_policies.copy_non_const_reference ) HTH -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From anderson.lizardo at openbossa.org Mon Jun 15 21:24:29 2009 From: anderson.lizardo at openbossa.org (Anderson Lizardo) Date: Mon, 15 Jun 2009 15:24:29 -0400 Subject: [C++-sig] boost.python: no overflow checking for unsigned int parameters ? Message-ID: <55b616790906151224g6ae9a712ke3dab8dfe3e1c556@mail.gmail.com> Hi, I noticed that boost.python silently accepts negative (or too long) integers for constructor/methods that have "unsigned int" parameters. (signed) "int" arguments seem to be handled just fine. Is it possible to tell boost.python to check for overflow and raise an exception just like it happens to int parameters ? BTW, I'm using boost1.38. See test case below (save it as "mymodule.cpp"), "MyClass1" uses int parameters, "MyClass2" uses unsigned int: ### snip ### /* $ g++ -shared -I/usr/include/python2.6 -o mymodule.so mymodule.cpp -lboost_python-mt $ python Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> from mymodule import MyClass1, MyClass2 >>> >>> MyClass1(840835495615213080).some_method(840835495615213080) Traceback (most recent call last): File "", line 1, in OverflowError: long int too large to convert to int >>> >>> MyClass1(-1).some_method(-1) MyClass1: -1 MyClass1::some_method: -1 >>> >>> MyClass2(840835495615213080).some_method(840835495615213080) MyClass2: 4294967295 MyClass2::some_method: 4294967295 >>> >>> MyClass2(-1).some_method(-1) MyClass2: 4294967295 MyClass2::some_method: 4294967295 >>> */ #include #include class MyClass1 { public: MyClass1(int x) { std::cout << "MyClass1: " << x << std::endl; } void some_method(int x) { std::cout << "MyClass1::some_method: " << x << std::endl; } }; class MyClass2 { public: MyClass2(unsigned int x) { std::cout << "MyClass2: " << x << std::endl; } void some_method(unsigned int x) { std::cout << "MyClass2::some_method: " << x << std::endl; } }; using namespace boost::python; BOOST_PYTHON_MODULE(mymodule) { class_("MyClass1", init()) .def("some_method", &MyClass1::some_method) ; class_("MyClass2", init()) .def("some_method", &MyClass2::some_method) ; } ### snip ### Regards, -- Anderson Lizardo OpenBossa Labs - INdT Manaus - Brazil From sipickles at googlemail.com Mon Jun 15 22:05:30 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Mon, 15 Jun 2009 21:05:30 +0100 Subject: [C++-sig] Avoiding copying non-copyables Message-ID: <4A36A98A.90807@googlemail.com> Hi, I have a c++ class which uses boost::thread and boost::mutex. I cannot expose this class directly with boost::python, since mutex is non-copyable. so I did this: /////////////////////////// C++ class ResourceManager {}; ResourceManager* ResourceManagerFactory() { return new ResourceManager(); } def("ResourceManagerFactory", ResourceManagerFactory, return_value_policy()); //////////////////// This compiles but I at python runtime I get: ##### PYTHON self.resourceManager = ResourceManagerFactory() TypeError: No Python class registered for C++ class class ResourceManager ##### Am I doing this wrong? Wrong return policy? Thanks Simon From rwgk at yahoo.com Mon Jun 15 22:23:57 2009 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Mon, 15 Jun 2009 13:23:57 -0700 (PDT) Subject: [C++-sig] boost.python: no overflow checking for unsigned int parameters ? In-Reply-To: <55b616790906151224g6ae9a712ke3dab8dfe3e1c556@mail.gmail.com> References: <55b616790906151224g6ae9a712ke3dab8dfe3e1c556@mail.gmail.com> Message-ID: <59161.36991.qm@web111401.mail.gq1.yahoo.com> If you send me a patch (against the boost trunk) I'll test it and check it in. Probably, the file to be changed is boost/libs/python/src/converter/builtin_converters.cpp I'd try working with boost/numeric/conversion/cast.hpp Ralf ----- Original Message ---- From: Anderson Lizardo To: cplusplus-sig at python.org Sent: Monday, June 15, 2009 12:24:29 PM Subject: [C++-sig] boost.python: no overflow checking for unsigned int parameters ? Hi, I noticed that boost.python silently accepts negative (or too long) integers for constructor/methods that have "unsigned int" parameters. (signed) "int" arguments seem to be handled just fine. Is it possible to tell boost.python to check for overflow and raise an exception just like it happens to int parameters ? BTW, I'm using boost1.38. See test case below (save it as "mymodule.cpp"), "MyClass1" uses int parameters, "MyClass2" uses unsigned int: ### snip ### From renatox at gmail.com Mon Jun 15 22:24:08 2009 From: renatox at gmail.com (Renato Araujo) Date: Mon, 15 Jun 2009 17:24:08 -0300 Subject: [C++-sig] Avoiding copying non-copyables In-Reply-To: <4A36A98A.90807@googlemail.com> References: <4A36A98A.90807@googlemail.com> Message-ID: <95291a80906151324u568234cen82afa85d22a52734@mail.gmail.com> Hi Simon, Did you export your class ResourceManager to python? If this class not is copyable you can export with some like that: class_("ClassName") BR Renato On Mon, Jun 15, 2009 at 5:05 PM, Simon Pickles wrote: > Hi, > > I have a c++ class which uses boost::thread and boost::mutex. > > I cannot expose this class directly with boost::python, since mutex is > non-copyable. > > so I did this: > > /////////////////////////// C++ > class ResourceManager > {}; > > ResourceManager* ResourceManagerFactory() > { > ? ? ? ?return new ResourceManager(); > } > > def("ResourceManagerFactory", ResourceManagerFactory, > return_value_policy()); > > //////////////////// > > > This compiles but I at python runtime I get: > > ##### PYTHON > self.resourceManager = ResourceManagerFactory() > TypeError: No Python class registered for C++ class class ResourceManager > > ##### > > Am I doing this wrong? Wrong return policy? > > Thanks > > Simon > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Renato Araujo Oliveira Filho From nutrina9 at gmail.com Mon Jun 15 23:22:25 2009 From: nutrina9 at gmail.com (Gerald Iakobinyi-Pich) Date: Tue, 16 Jun 2009 00:22:25 +0300 Subject: [C++-sig] boost.python build failure on Mac Message-ID: <5485B164-29BD-41D3-9F01-BF5115A7BCC3@gmail.com> Hello, I am facing an error when trying to build boost.python. I hope I am in the right place to address this issue. If not please redirect me ... So first I ran bootstrap.sh in my boost root dir pro:boost_1_39_0, and that is the output: -n Building Boost.Jam with toolset darwin... tools/jam/src/bin.macosxx86/bjam -n Detecting Python version... 2.6 -n Detecting Python root... /Library/Frameworks/Python.framework/Versions/2.6 -n Unicode/ICU support for Boost.Regex?... not found. Backing up existing Boost.Build configuration in project-config.jam.4 Generating Boost.Build configuration in project-config.jam... Bootstrapping is done. To build, run: ./bjam To adjust configuration, edit 'project-config.jam'. Further information: - Command line help: ./bjam --help - Getting started guide: http://www.boost.org/more/getting_started/unix-variants.html - Boost.Build documentation: http://www.boost.org/boost-build2/doc/html/index.html No I tried to build the 'quickstart' example, and the linker seems to enter some kind of 'endless' loop. Building extending.so, never comes to an end: .... darwin.compile.c++ /Users/nutrina/temp/boost/bin.v2/libs/python/build/ darwin-4.0.1/debug/import.o darwin.compile.c++ /Users/nutrina/temp/boost/bin.v2/libs/python/build/ darwin-4.0.1/debug/exec.o darwin.compile.c++ /Users/nutrina/temp/boost/bin.v2/libs/python/build/ darwin-4.0.1/debug/object/function_doc_signature.o darwin.link.dll /Users/nutrina/temp/boost/bin.v2/libs/python/build/ darwin-4.0.1/debug/libboost_python-xgcc40-d-1_39.dylib darwin.link.dll /Users/nutrina/temp/boost-python-quickstart/ darwin-4.0.1/debug/extending.so Has somebody else encountered the same error? Could somebody give some hint? Btw. I am using XCode 3.1.2 + the toolset it comes with (i686- apple-darwin9-gcc-4.0.1 (GCC) 4.0.1 (Apple Inc. build 5490)). Thanks, Gerald From seefeld at sympatico.ca Tue Jun 16 00:00:53 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Mon, 15 Jun 2009 18:00:53 -0400 Subject: [C++-sig] Fw: g++ compiler limitations other than -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? In-Reply-To: <003a01c9eaf2$91a27c60$5f8a1093@dhcp.ess.us.ray.com> References: <003a01c9eaf2$91a27c60$5f8a1093@dhcp.ess.us.ray.com> Message-ID: <4A36C495.7030707@sympatico.ca> On 06/11/2009 08:13 PM, Christopher A Mejia wrote: > Stefan / cplusplus-sig, > > Based on Stefan's encouragement, I developed a minimal complete test case Thanks ! Can you please submit that as an issue on https://svn.boost.org/trac/boost/ ? We may then follow-up there... Stefan -- ...ich hab' noch einen Koffer in Berlin... From camejia at raytheon.com Tue Jun 16 06:10:39 2009 From: camejia at raytheon.com (Christopher A Mejia) Date: Mon, 15 Jun 2009 21:10:39 -0700 Subject: [C++-sig] Fw: g++ compiler limitations otherthan -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? References: <003a01c9eaf2$91a27c60$5f8a1093@dhcp.ess.us.ray.com> <4A36C495.7030707@sympatico.ca> Message-ID: <008b01c9ee38$68ac9590$5f8a1093@dhcp.ess.us.ray.com> Stefan, OK--it's in the system as #3183. Thanks again for your help, and anyone should feel free to let me know if they need more information about this issue. --Chris ----- Original Message ----- From: Stefan Seefeld To: Development of Python/C++ integration Sent: Monday, June 15, 2009 3:00 PM Subject: Re: [C++-sig] Fw: g++ compiler limitations otherthan -ftemplate-depth-n and -DBOOST_PYTHON_MAX_ARITY ? On 06/11/2009 08:13 PM, Christopher A Mejia wrote: > Stefan / cplusplus-sig, > > Based on Stefan's encouragement, I developed a minimal complete test case Thanks ! Can you please submit that as an issue on https://svn.boost.org/trac/boost/ ? We may then follow-up there... Stefan -- ...ich hab' noch einen Koffer in Berlin... From sipickles at googlemail.com Thu Jun 18 09:49:01 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Thu, 18 Jun 2009 08:49:01 +0100 Subject: [C++-sig] boost::python and __declspec(align(16)) Message-ID: <4A39F16D.10108@googlemail.com> Hi, I think this is an MSVC problem caused by the bad STL implementation, but I wondered if there was a work around. Both my client (Win32 MSVC8) and server (ubuntu GCC 4.2.3) share the same c++ code. It compiles and runs on GCC but fails on MSVC. This uses the the bullet physics library and includes a type defined like this: __declspec(align(16)) btVector3 {/*...*/}; When building my boost::python extension with: class_("BtVector3", init()); I get an error like this : C:\boost_1_38_0\boost/python/converter/as_to_python_function.hpp(21) : error C2719: 'unnamed-parameter': formal parameter with __declspec(align('16')) won't be aligned C:\boost_1_38_0\boost/python/to_python_converter.hpp(88) : see reference to class template instantiation 'boost::python::converter::as_to_python_function' being compiled with [ T=btVector3, /* .... Huge call stack omitted for clarity.... */ build.cpp(155) : see reference to class template instantiation 'boost::python::class_' being compiled with [ W=btVector3 ] Is there a way to overcome this? switch to MinGW? :) Thank you very much Simon From anderson.lizardo at openbossa.org Thu Jun 18 15:08:20 2009 From: anderson.lizardo at openbossa.org (Anderson Lizardo) Date: Thu, 18 Jun 2009 09:08:20 -0400 Subject: [C++-sig] boost.python: no overflow checking for unsigned int parameters ? In-Reply-To: <59161.36991.qm@web111401.mail.gq1.yahoo.com> References: <55b616790906151224g6ae9a712ke3dab8dfe3e1c556@mail.gmail.com> <59161.36991.qm@web111401.mail.gq1.yahoo.com> Message-ID: <55b616790906180608r7b7282b5s750a5cff625a5f4@mail.gmail.com> On Mon, Jun 15, 2009 at 4:23 PM, Ralf W. Grosse-Kunstleve wrote: > > If you send me a patch (against the boost trunk) I'll test it and check it in. > Probably, the file to be changed is > ?boost/libs/python/src/converter/builtin_converters.cpp > I'd try working with > ?boost/numeric/conversion/cast.hpp Done. See https://svn.boost.org/trac/boost/ticket/3189 and the patch attached to it. Note the patch still lacks the addition of a test to the test suite. I'm working on it, but you can test it in advance. Regards, -- Anderson Lizardo OpenBossa Labs - INdT Manaus - Brazil From mikeowen at llnl.gov Fri Jun 19 02:58:20 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Thu, 18 Jun 2009 17:58:20 -0700 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> Message-ID: <99E6647C-40C8-460D-8B39-3351C6F87015@llnl.gov> Hello Gustavo and Taner, Sorry I didn't follow up on this earlier with you guys -- I've been on travel for a while since I started looking at pybindgen and just returned. After Gustavo pointed out the python documentation for the sequence method slots, I thought I would take a look at modifying pybindgen to allow me to fill in the sequence slots for CppClass much like the numeric operator slots are handled. I've been able to do this, and am attaching some modified pybindgen source to this message for your perusal. With these modifications I can pretty easily expose std::vector so that length, get and set item (index access), and iteration work. I'll fill in more methods as needed, but this is a start. I took this tack 'cause in my code I have C++ objects that naturally obey both numeric object protocols (such as +, -, etc) as well as sequence methods, and I need to reflect those operations into python. I want them to look like python objects as much as possible, so it's nice to be able to use [] rather than () for indexing for instance (it's best not to confuse your users more than necessary). :) The way I have this working is if the user adds any of the following methods to a CppClass ("__len__", "__getitem__", "__setitem__"), then the corresponding slots in the sequence object protocols are filled in in using those methods. I think this is fairly natural, since this is how you add sequence methods to an object you define in python natively. With my modifications I can add a container as a cppclass like so: mod = Module("example") mod.add_include("") mod.add_include('"example.hh"') std = mod.add_cpp_namespace("std") vecint = std.add_class("vector", template_parameters=["int"], custom_template_class_name="vector_of_int") vecint.add_constructor([]) vecint.add_constructor([param("int", "size")]) vecint.add_constructor([param("int", "size"), param("int", "value")]) vecint.add_method("size", "int", [], custom_name = "__len__") vecint.add_function_as_method("indexContainer", "int", [param("std::vector", "self"), param("int", "index")], template_parameters = ["std::vector"], custom_name = "__getitem__") vecint.add_function_as_method("assignToPosition", None, [param("std::vector", "self"), param("int", "index"), param("int", "value")], template_parameters = ["std::vector"], custom_name = "__setitem__") vecint.add_method("push_back", None, [param("int", "value")], custom_name="append") where "example.hh" contains a few helper methods: #include namespace std { //------------------------------------------------------------------------------ // Extract a value from a container. //------------------------------------------------------------------------------ template inline typename Container::value_type& indexContainer(Container& container, const size_t index) { try { return container.at(index); } catch (out_of_range) { PyErr_SetString(PyExc_IndexError, "Container index out of range"); } } //------------------------------------------------------------------------------ // Assign to a postion in a container. //------------------------------------------------------------------------------ template inline void assignToPosition(Container& container, const size_t index, const typename Container::value_type& value) { if (index >= container.size()) { PyErr_SetString(PyExc_IndexError, "Container index out of range"); } else { container[index] = value; } } } And the resulting vector can be used from python kind of like you would naturally expect: alastor1{owen}77: python Python 2.6.1 (r261:67515, Jun 2 2009, 15:40:32) [GCC 4.3.2 20081007 (Red Hat 4.3.2-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import example >>> v = example.std.vector_of_int(20, 5) >>> len(v) 20 >>> [x for x in v] [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5] >>> v[10] = 100 >>> [x for x in v] [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 100, 5, 5, 5, 5, 5, 5, 5, 5, 5] >>> In this process I've also made a few other modifications to pybindgen: 1. In CppClass I've generalized the binary_numeric_operators to allow the user to specify python types as one of the arguments. I needed to do this so I could allow types to define "*" with a python float for instance, rather than only allowing operators with other CppClass wrapped objects. 2. I've also added inplace_numeric_operators as an option for CppClass, which fills in the appropriate slots in the number methods struct. 3. I added name mangling for template arguments, since some of my template arguments for methods are themselves templates (such as in the example above). The changes I have are made against pybindgen-0.10.0, and you should compare the files in this tarball to that version. I'm no Python C- API whiz, so I won't claim these are the most optimal ways to handle these extensions to pybindgen, but it does work and passes all of pybindgen's internal tests. I'm continuing to look at how I have to modify pybindgen in order to meet the needs of wrapping my code, but I don't want to wander too far afield of the mainline release! If these kinds of mods are not going to work for want you want to do Gustavo, please let me know! Anyway, check out these mods as you can and let me know what you think. Mike. On Jun 5, 2009, at 10:00 AM, Gustavo Carneiro wrote: > > > 2009/6/5 J. Michael Owen > Hi Taner, > > Thanks for the suggestion! I actually was trying the same solution, > except I wasn't sure how to handle the iterators. I'll try it out > as you do in your example -- are you able to iterate over the vector > in python the usual way? As in "for x in vec:"? > > Two curious things I noted about wrapping "std::vector" myself: > > 1. I can't seem to get indexing access to work, even though I wrap > an indexing operation with the python name "__getitem__" > > 2. I also can't get python's len function to work, even if I > provide "__len__" as follows: > veci.add_method("size", "int", [], custom_name="__len__") > > Have you been able to make either of those methods work? > > It won't work because those are special methods in C/C++ side; > Python extension types use a "slots" mechanism for this kind of > thing, see http:// www. python.org/doc/2.6/c-api/typeobj.html > > > Thanks! > > Mike. > > -- > Gustavo J. A. M. Carneiro > INESC Porto, Telecommunications and Multimedia Unit > "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: pybindgen-0.10.0-mods.tbz Type: application/octet-stream Size: 33517 bytes Desc: not available URL: -------------- next part -------------- An HTML attachment was scrubbed... URL: From sipickles at googlemail.com Fri Jun 19 09:30:37 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Fri, 19 Jun 2009 08:30:37 +0100 Subject: [C++-sig] Boost.Python and STLPort, ArgumentError Message-ID: <4A3B3E9D.1090109@googlemail.com> Hello, Configuration: WinXP SP3 MSVC8 Python 2.5.4 Boost 1.38 STLPort 5.2 Bullet Physics (requires STLPort to avoid Alignment problems) I need to use STLPort with my c++ extension modules, due to Bullet Physics API. I've managed to build STLPort, rebuilt boost (bjam stllib=stlport stage), and rebuild my project (bjam define=__STL_DEBUG). My problem starts when I run my python code. This calls c++ modules, and I get a type mismatch: E:\WorkingCopies\client\extended>python client.py Traceback (most recent call last): File "client.py", line 232, in log.StartLogToFile("Log_Client.txt") Boost.Python.ArgumentError: Python argument types in None.None(Logger, str) did not match C++ signature: None(class Logger {lvalue}, class stlpd_std::basic_string,class stlpd_std::allocator >) Clearly, the c++ module, built with stlport, is expecting an stlp::string while the calling python code is providing a std::string (eg a python string) Please don't tell me I have to rebuild python using stlport? I have not had much success building python from source in the past! :) Is there a simpler remedy for this hopefully small hurdle? Thank you for your help Simon From nickm at sitius.com Fri Jun 19 16:11:24 2009 From: nickm at sitius.com (Nikolay Mladenov) Date: Fri, 19 Jun 2009 10:11:24 -0400 Subject: [C++-sig] Boost.Python and STLPort, ArgumentError In-Reply-To: <4A3B3E9D.1090109@googlemail.com> References: <4A3B3E9D.1090109@googlemail.com> Message-ID: It looks like your Boost.Python requires rebuild with stlport? The error message comes from Boost.Python (function.cpp I think). It seems to me that your module is build with stlport but boost_python is not. HTH, Nikolay Simon Pickles wrote: > Hello, > > Configuration: > WinXP SP3 > MSVC8 > Python 2.5.4 > Boost 1.38 > STLPort 5.2 > Bullet Physics (requires STLPort to avoid Alignment problems) > > I need to use STLPort with my c++ extension modules, due to Bullet > Physics API. > > I've managed to build STLPort, rebuilt boost (bjam stllib=stlport > stage), and rebuild my project (bjam define=__STL_DEBUG). My problem > starts when I run my python code. This calls c++ modules, and I get a > type mismatch: > > E:\WorkingCopies\client\extended>python client.py > Traceback (most recent call last): > File "client.py", line 232, in > log.StartLogToFile("Log_Client.txt") > Boost.Python.ArgumentError: Python argument types in > None.None(Logger, str) > did not match C++ signature: > None(class Logger {lvalue}, class stlpd_std::basic_string stlpd_std::char_traits,class stlpd_std::allocator >) > > Clearly, the c++ module, built with stlport, is expecting an > stlp::string while the calling python code is providing a std::string > (eg a python string) > > Please don't tell me I have to rebuild python using stlport? I have not > had much success building python from source in the past! :) > > Is there a simpler remedy for this hopefully small hurdle? > > Thank you for your help > > Simon From sipickles at googlemail.com Fri Jun 19 17:06:11 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Fri, 19 Jun 2009 16:06:11 +0100 Subject: [C++-sig] Boost.Python and STLPort, ArgumentError In-Reply-To: References: <4A3B3E9D.1090109@googlemail.com> Message-ID: <4A3BA963.4010308@googlemail.com> Hello Nikolay, Thanks for the reply. I _think_ it is built with stlport. I rebuilt boost.python with my user-config.jam changed and the stdlib=stlport option. It generated new libraries with the 'p' suffix (for stlport) When I bjammed my c++ extension module, I had to link to the boost_python-xxx-gdp.lib files and it required the dll counterparts to build and run.... Perhaps I need to build my project with: bjam define=__STL_DEBUG stdlib=stlport I will try on the train home :) Thanks Simon Nikolay Mladenov wrote: > It looks like your Boost.Python requires rebuild with stlport? > The error message comes from Boost.Python (function.cpp I think). > It seems to me that your module is build with stlport but boost_python > is not. > > HTH, > > Nikolay > > > Simon Pickles wrote: >> Hello, >> >> Configuration: >> WinXP SP3 >> MSVC8 >> Python 2.5.4 >> Boost 1.38 >> STLPort 5.2 >> Bullet Physics (requires STLPort to avoid Alignment problems) >> >> I need to use STLPort with my c++ extension modules, due to Bullet >> Physics API. >> >> I've managed to build STLPort, rebuilt boost (bjam stllib=stlport >> stage), and rebuild my project (bjam define=__STL_DEBUG). My problem >> starts when I run my python code. This calls c++ modules, and I get a >> type mismatch: >> >> E:\WorkingCopies\client\extended>python client.py >> Traceback (most recent call last): >> File "client.py", line 232, in >> log.StartLogToFile("Log_Client.txt") >> Boost.Python.ArgumentError: Python argument types in >> None.None(Logger, str) >> did not match C++ signature: >> None(class Logger {lvalue}, class >> stlpd_std::basic_string,class >> stlpd_std::allocator >) >> >> Clearly, the c++ module, built with stlport, is expecting an >> stlp::string while the calling python code is providing a std::string >> (eg a python string) >> >> Please don't tell me I have to rebuild python using stlport? I have >> not had much success building python from source in the past! :) >> >> Is there a simpler remedy for this hopefully small hurdle? >> >> Thank you for your help >> >> Simon > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > From josh.davidson at lmco.com Fri Jun 19 22:54:19 2009 From: josh.davidson at lmco.com (Davidson, Josh) Date: Fri, 19 Jun 2009 14:54:19 -0600 Subject: [C++-sig] pygccxml and multidimensional arrays In-Reply-To: <7465b6170905190333p6c1262b4hd63899b902a81179@mail.gmail.com> References: <5A2986C73A1DCC4EB5CC1C22D82700090FD12E26@emss02m11.us.lmco.com> <7465b6170904152204q5e0381cdmb6643e3bc5de918b@mail.gmail.com> <5A2986C73A1DCC4EB5CC1C22D82700090FDAB895@emss02m11.us.lmco.com> <5A2986C73A1DCC4EB5CC1C22D82700090FF26840@emss02m11.us.lmco.com> <7465b6170904231011xf63c219ge7f398e70ff1bfbd@mail.gmail.com> <5A2986C73A1DCC4EB5CC1C22D82700090FF61E05@emss02m11.us.lmco.com> <7465b6170904231121o17cee0acm47d847328c3d1dce@mail.gmail.com> <7465b6170904231123x7c7fbb73j9b95aee05786c757@mail.gmail.com> <7465b6170905190333p6c1262b4hd63899b902a81179@mail.gmail.com> Message-ID: <5A2986C73A1DCC4EB5CC1C22D827000910D6991B@emss02m11.us.lmco.com> Yes, your patch corrected the problem. Is the next official release scheduled? Thanks, Josh -----Original Message----- From: cplusplus-sig-bounces+josh.davidson=lmco.com at python.org [mailto:cplusplus-sig-bounces+josh.davidson=lmco.com at python.org] On Behalf Of Roman Yakovenko Sent: Tuesday, May 19, 2009 4:34 AM To: Development of Python/C++ integration Subject: Re: [C++-sig] pygccxml and multidimensional arrays On Thu, Apr 23, 2009 at 9:23 PM, Roman Yakovenko wrote: > On Thu, Apr 23, 2009 at 9:21 PM, Roman Yakovenko > wrote: >> On Thu, Apr 23, 2009 at 8:17 PM, Davidson, Josh wrote: >>> I'm using information provided by pygccxml to generate Construct objects >>> (http://construct.wikispaces.com/) that are capable of reading and >>> writing to data structures stored in shared memory. ?Right now, I just >>> manually reverse the indices any time I detect an array type before I >>> generate a Construct. >> >> Okey. > > Sorry, I forgot to add, can you open bug on SourceForge project page? Josh, I think the bug is fixed now: http://pygccxml.svn.sourceforge.net/viewvc/pygccxml?view=rev&revision=1735 Can you verify this? Thanks -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org http://mail.python.org/mailman/listinfo/cplusplus-sig From cschramm at shakaweb.org Sun Jun 21 13:40:07 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Sun, 21 Jun 2009 13:40:07 +0200 Subject: [C++-sig] Function handling with Boost Message-ID: <4A3E1C17.7070502@shakaweb.org> Hi, I'm new to Python's C API and developing my very first project using Boost::Python (bpy). My goal is executing user defined scripts within an embedded python environment using custom stdout and stderr and providing a special built in module. Beside a bunch of classes (which can be defined using bpy quite comfortable) the module shall contain a couple of functions and since bpy is apparently not primarily targeted at embedding and doesn't provide a function type this is the point where all my problems start. My current solution is to get the global scope: global = import("__main__").attr("__dict__"); And add my module using Py_InitModule3: global["my_mod"] = object(handle<>(borrowed(Py_InitModule3("my_mod", functions, "docstr")))); Adding classes and there methods is a nice, clear and easy task: global["my_mod"].attr("__dict__")["myclass"] = class_("myclass", bpy::init<>()) .def("f1", &CPPClass::f1) .def("f2", &CPPClass::f2); But the functions need to be in the functions parameter of Py_InitModule3, defined like: PyMethodDef functions[] = { {"f1", &f1, METH_VARARGS, "my_doc"}, {"f2", &f2, METH_VARARGS, "my_doc"}, {0, 0} }; And now the implementations have to be PyCFunction type, i. e. have the signature PyObject* (PyObject*, PyObject*), not a (more or less) arbitrary one they could have when used in bpy's def(). This means the number of arguments given to the function within the python code could be wrong, I need to convert from PyObject to bpy types to C++ types and all the way back and there are traps like returning temporary pointers or memory leaks. An implementation could look like: PyObject* f1(PyObject* self, PyObject* args) { bpy::handle<> arg1, arg2; PyArg_UnpackTuple(args, "func", 1, 2, &arg1, &arg2); if (arg2.get()) { ... } ... ...bpy::extract(bpy::object(arg1))... return Py_BuildValue("s", my_cstr_result); } Beside the problems already mentioned this means missing bpy features for the functions like automatic docstrings, return value policies etc. So I would really like to know whether there's a better way to do what I'm doing. Especially a more boost like way ;-) Thanks for any reply! Christopher Schramm From jesse_lehrman at animationlab.com Mon Jun 22 08:52:35 2009 From: jesse_lehrman at animationlab.com (Jesse Lehrman) Date: Mon, 22 Jun 2009 09:52:35 +0300 Subject: [C++-sig] Differences between 32-bit and 64-bit python25.lib Message-ID: <1FF132D810C1AC409FD8A47BDC96E3DB01CFBB7C2D@alabex01> I'm trying to compile SIP/PyQT 4.5 for Windows Python 64-bit. I'm running into an error when attempting to link the SIP/PyQT object files against python25.lib. The errors are "can't find reference..." to symbols that should be in python25.lib. I tried using the 32-bit version of python25.lib and it worked. This led me to look at python25.lib using dumpbin.exe. I'm using the install binaries from www.python.org. It seems that most of the exports in the 32-bit version have a preceding underscore while in the 64-bit version they don't. For example: python25.lib 32-bit: _PyFloat_AsDouble _PyFloat_AsReprString _PyFloat_AsString _PyFloat_Fini _PyFloat_FromDouble _PyFloat_FromString _PyFloat_Type python25.lib 64-bit: PyFloat_AsDouble PyFloat_AsReprString PyFloat_AsString PyFloat_Fini PyFloat_FromDouble PyFloat_FromString PyFloat_Type Does anybody have an idea of why this is done and how I can get around it? Thank you, Jesse ________________________________ Please consider the environment before printing this email. -------------- next part -------------- An HTML attachment was scrubbed... URL: From zao at acc.umu.se Mon Jun 22 10:37:21 2009 From: zao at acc.umu.se (Lars Viklund) Date: Mon, 22 Jun 2009 10:37:21 +0200 Subject: [C++-sig] Differences between 32-bit and 64-bit python25.lib In-Reply-To: <1FF132D810C1AC409FD8A47BDC96E3DB01CFBB7C2D@alabex01> References: <1FF132D810C1AC409FD8A47BDC96E3DB01CFBB7C2D@alabex01> Message-ID: <20090622083721.GA14783@suiko.acc.umu.se> On Mon, Jun 22, 2009 at 09:52:35AM +0300, Jesse Lehrman wrote: > It seems that most of the exports in the 32-bit version have a preceding underscore while in the 64-bit version they don't. For example: > > python25.lib 32-bit: > _PyFloat_AsDouble > > python25.lib 64-bit: > PyFloat_AsDouble > > Does anybody have an idea of why this is done and how I can get around it? x86 has a whole slew of calling conventions, and in order to disambiguate between them, symbols are decorated according to the convention used [1]. In this case, a leading underscore means it's cdecl. With x86_64, there's a new and shiny calling convention [2], replacing all the old ones. As there's only one, there's no need to decorate that one with anything in a symbol, thus the lack of _ in your 64-bit symbol. This is all from a Windows perspective of course, as that's your context. As for getting things to do the right thing, I'm unsure. [1] http://blogs.msdn.com/oldnewthing/archive/2004/01/08/48616.aspx [2] http://blogs.msdn.com/freik/archive/2005/03/17/398200.aspx -- Lars Viklund | zao at acc.umu.se | 070-310 47 07 From yuval.nahmany at gmail.com Mon Jun 22 13:06:25 2009 From: yuval.nahmany at gmail.com (Yuval Nahmany) Date: Mon, 22 Jun 2009 14:06:25 +0300 Subject: [C++-sig] embedding python in C++: exceptions, a newbie Q Message-ID: Hi, I am trying to use boost.python to wrap some classes in an application, and have python embedded in my app and make some instances from the application accessible thru simple script. I got the impression that with boost python it should be quite simple for basics, but I cannot have a simple code as bellow run: 1. If I uncomment the //import("MyModule"); I get a runtime error - and an application crash. And without it- I get an exception that the C++ class CMyWrap is not associated with a python class. 2. If I just try to do a simpler thing like //exec("dir(__main__)", main_namespace); I get "bad argument to an internal function" as an exception. I use python26_d.dll/ python26_d.lib and libboost_python-vc90-mt-1_39.lib (boost 1.39 compiled as static lib). Can you please help? Can you refer me to any documentation that may help? TIA, Yuv. class CMyWrap { public: string Greet() { return "I'm the dude!?"; } }; class CppClass { public: int getNum() { return 7; } }; BOOST_PYTHON_MODULE(MyModule) { class_("CMyWrap") .def("greet", &CMyWrap::Greet); } int _tmain(int argc, TCHAR* argv[], TCHAR* envp[]) { int nRetCode = 0; CMyWrap myObj; try { PyImport_AppendInittab( "MyModule", &initMyModule); Py_Initialize(); object main_module = import("__main__"); object main_namespace = main_module.attr("__dict__"); //import("MyModule"); //exec("dir(__main__)", main_namespace); main_namespace["myObj"] = ptr(&myObj); } catch( error_already_set ) { PyErr_Print(); } Py_Finalize(); return nRetCode; } -- Yuval Nahmany cell. 054-2020665 From nicolaslara at gmail.com Mon Jun 22 13:31:53 2009 From: nicolaslara at gmail.com (Nicolas Lara) Date: Mon, 22 Jun 2009 11:31:53 +0000 Subject: [C++-sig] wchar_t* type conversion In-Reply-To: <46ef4bec0906210806tb5cd894q7dcd5fdd42eeab8e@mail.gmail.com> References: <46ef4bec0906210806tb5cd894q7dcd5fdd42eeab8e@mail.gmail.com> Message-ID: <46ef4bec0906220431s4f7899dflca213c20c85c96e1@mail.gmail.com> Hello, I'm new to Boost::Python and am encountering a problem that seem to be quite common: I need to expose a class to C++ whose constructor takes a wchar_t* as a parameter. The problem is that there is no default type converter from wchar_t to python and vice-versa. I wanted to know if there was a standard way of doing this. Or if anyone could point me to an implementation of this type conversion. So far I'm doing this (based on the faq): struct wchar_t_to_python_str { ?static PyObject* convert(wchar_t* const& w) ?{ ? ?string s = ""; ? ?wchar_t* w2 = w; ? ?while(*w2++ != 0){ ? ? ?s += *w2; ? ?} ? ?return boost::python::incref(boost::python::object(s).ptr()); ?} }; struct wchar_t_from_python_str { ?wchar_t_from_python_str() ?{ ? ?boost::python::converter::registry::push_back( ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); ?} ?static void* convertible(PyObject* obj_ptr) ?{ ? ?if (!PyString_Check(obj_ptr)) return 0; ? ?return obj_ptr; ?} ?static void construct( ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, boost::python::converter::rvalue_from_python_stage1_data* data) ?{ ? ?const char* value = PyString_AsString(obj_ptr); ? ?Py_ssize_t l = PyString_Size(obj_ptr); ? ?if (value == 0) boost::python::throw_error_already_set(); ? ?void* storage = ( (boost::python::converter::rvalue_from_python_storage*) ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; ? ?wchar_t* w = new wchar_t[l]; ? ?for(int i=0;iconvertible = w; ?} }; ?void init_module() ?{ ? ?to_python_converter(); ? ?wchar_t_from_python_str(); ? } BOOST_PYTHON_MODULE(queryparser){ ?init_module(); } But it doesn't seem to be working. It compiles, but python doesn't understand the conversion. Any thoughts? Thanks in advance! -- Nicolas Lara Linux user #380134 http://nicolas-lara.blogspot.com/ Public key id: 0x152e7713 at http://subkeys.pgp.net/ From renatox at gmail.com Mon Jun 22 15:48:54 2009 From: renatox at gmail.com (Renato Araujo) Date: Mon, 22 Jun 2009 10:48:54 -0300 Subject: [C++-sig] wchar_t* type conversion In-Reply-To: <46ef4bec0906220431s4f7899dflca213c20c85c96e1@mail.gmail.com> References: <46ef4bec0906210806tb5cd894q7dcd5fdd42eeab8e@mail.gmail.com> <46ef4bec0906220431s4f7899dflca213c20c85c96e1@mail.gmail.com> Message-ID: <95291a80906220648y36ce918amea421d100b8d5549@mail.gmail.com> Hi Nicolas, Can you send a simple example, where the convention fail (c++ to python or python to c++)? What kind of test you are running? BR Renato On Mon, Jun 22, 2009 at 8:31 AM, Nicolas Lara wrote: > Hello, > I'm new to Boost::Python and am encountering a problem that seem to be > quite common: I need to expose a class to C++ whose constructor takes > a wchar_t* as a parameter. > The problem is that there is no default type converter from wchar_t to > python and vice-versa. > > I wanted to know if there was a standard way of doing this. Or if > anyone could point me to an implementation of this type conversion. > > So far I'm doing this (based on the faq): > > struct wchar_t_to_python_str > { > ?static PyObject* convert(wchar_t* const& w) > ?{ > ? ?string s = ""; > ? ?wchar_t* w2 = w; > ? ?while(*w2++ != 0){ > ? ? ?s += *w2; > ? ?} > ? ?return boost::python::incref(boost::python::object(s).ptr()); > ?} > }; > > struct wchar_t_from_python_str > { > ?wchar_t_from_python_str() > ?{ > ? ?boost::python::converter::registry::push_back( > ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, > ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, > ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); > ?} > > ?static void* convertible(PyObject* obj_ptr) > ?{ > ? ?if (!PyString_Check(obj_ptr)) return 0; > ? ?return obj_ptr; > ?} > > ?static void construct( > ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, > > boost::python::converter::rvalue_from_python_stage1_data* data) > ?{ > ? ?const char* value = PyString_AsString(obj_ptr); > ? ?Py_ssize_t l = PyString_Size(obj_ptr); > ? ?if (value == 0) boost::python::throw_error_already_set(); > ? ?void* storage = ( > > (boost::python::converter::rvalue_from_python_storage*) > ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; > > ? ?wchar_t* w = new wchar_t[l]; > ? ?for(int i=0;i ? ? ?w[i] = value[i]; > ? ?} > ? ?data->convertible = w; > ?} > }; > ?void init_module() > ?{ > ? ?to_python_converter(); > ? ?wchar_t_from_python_str(); > ? } > > BOOST_PYTHON_MODULE(queryparser){ > ?init_module(); > } > > But it doesn't seem to be working. It compiles, but python doesn't > understand the conversion. > Any thoughts? > > > Thanks in advance! > > -- > Nicolas Lara > Linux user #380134 > http://nicolas-lara.blogspot.com/ > Public key id: 0x152e7713 at http://subkeys.pgp.net/ > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Renato Araujo Oliveira Filho From nicolaslara at gmail.com Mon Jun 22 19:27:08 2009 From: nicolaslara at gmail.com (Nicolas Lara) Date: Mon, 22 Jun 2009 17:27:08 +0000 Subject: [C++-sig] wchar_t* type conversion In-Reply-To: <95291a80906220648y36ce918amea421d100b8d5549@mail.gmail.com> References: <46ef4bec0906210806tb5cd894q7dcd5fdd42eeab8e@mail.gmail.com> <46ef4bec0906220431s4f7899dflca213c20c85c96e1@mail.gmail.com> <95291a80906220648y36ce918amea421d100b8d5549@mail.gmail.com> Message-ID: <46ef4bec0906221027l3ed677e0x28162495332fdab@mail.gmail.com> Hi Renato, Thanks for the reply. Here's some test code. (I pasted it to make it more friendly =) http://paste.pocoo.org/show/124521/ Testing it in python: http://paste.pocoo.org/show/124523/ Thanks in advance! Best Regards For the archives: === test.cpp #include "CLucene.h" #include #include using namespace boost::python; struct wchar_t_to_python_str { static PyObject* convert(wchar_t* const& w) { string s = ""; wchar_t* w2 = w; while(*w2++ != 0){ s += *w2; } return boost::python::incref(boost::python::object(s).ptr()); } }; struct wchar_t_from_python_str { wchar_t_from_python_str() { boost::python::converter::registry::push_back( &convertible, &construct, boost::python::type_id()); } static void* convertible(PyObject* obj_ptr) { if (!PyString_Check(obj_ptr)) return 0; return obj_ptr; } static void construct( PyObject* obj_ptr, boost::python::converter::rvalue_from_python_stage1_data* data) { const char* value = PyString_AsString(obj_ptr); Py_ssize_t l = PyString_Size(obj_ptr); if (value == 0) boost::python::throw_error_already_set(); void* storage = ( (boost::python::converter::rvalue_from_python_storage*) data)->storage.bytes; wchar_t* w = new wchar_t[l]; for(int i=0;iconvertible = w; } }; void init_module() { to_python_converter(); wchar_t_from_python_str(); } struct Test { Test(){} wchar_t * testReturnValue(string s){ wchar_t *p=new wchar_t[s.size()]; for(string::size_type i=0; i("Test") //.def("testReturnValue", &Test::testReturnValue) // <- desn't compile .def("testArgument", &Test::testArgument); ; } ==== testing it in python >>> import test >>> t = test.Test() >>> t.testArgument(u"") --------------------------------------------------------------------------- ArgumentError Traceback (most recent call last) /home/.../build/ in () ArgumentError: Python argument types in Test.testArgument(Test, unicode) did not match C++ signature: testArgument(Test {lvalue}, wchar_t*) >>> t.testArgument("") --------------------------------------------------------------------------- ArgumentError Traceback (most recent call last) /home/.../build/ in () ArgumentError: Python argument types in Test.testArgument(Test, str) did not match C++ signature: testArgument(Test {lvalue}, wchar_t*) >>> from ctypes import c_wchar_p >>> c_wchar_p("hello") >>> c_wchar_p(u'hello') >>> w = c_wchar_p("hello") >>> t.testArgument(w) --------------------------------------------------------------------------- ArgumentError Traceback (most recent call last) /home/nicolas/devel/CyLucene/build/ in () ArgumentError: Python argument types in Test.testArgument(Test, c_wchar_p) did not match C++ signature: testArgument(Test {lvalue}, wchar_t*) On Mon, Jun 22, 2009 at 1:48 PM, Renato Araujo wrote: > Hi Nicolas, > > Can ?you send a simple example, where the convention fail (c++ to > python or python to c++)? What kind of test you are running? > > BR > Renato > > > > On Mon, Jun 22, 2009 at 8:31 AM, Nicolas Lara wrote: >> Hello, >> I'm new to Boost::Python and am encountering a problem that seem to be >> quite common: I need to expose a class to C++ whose constructor takes >> a wchar_t* as a parameter. >> The problem is that there is no default type converter from wchar_t to >> python and vice-versa. >> >> I wanted to know if there was a standard way of doing this. Or if >> anyone could point me to an implementation of this type conversion. >> >> So far I'm doing this (based on the faq): >> >> struct wchar_t_to_python_str >> { >> ?static PyObject* convert(wchar_t* const& w) >> ?{ >> ? ?string s = ""; >> ? ?wchar_t* w2 = w; >> ? ?while(*w2++ != 0){ >> ? ? ?s += *w2; >> ? ?} >> ? ?return boost::python::incref(boost::python::object(s).ptr()); >> ?} >> }; >> >> struct wchar_t_from_python_str >> { >> ?wchar_t_from_python_str() >> ?{ >> ? ?boost::python::converter::registry::push_back( >> ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, >> ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, >> ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); >> ?} >> >> ?static void* convertible(PyObject* obj_ptr) >> ?{ >> ? ?if (!PyString_Check(obj_ptr)) return 0; >> ? ?return obj_ptr; >> ?} >> >> ?static void construct( >> ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, >> >> boost::python::converter::rvalue_from_python_stage1_data* data) >> ?{ >> ? ?const char* value = PyString_AsString(obj_ptr); >> ? ?Py_ssize_t l = PyString_Size(obj_ptr); >> ? ?if (value == 0) boost::python::throw_error_already_set(); >> ? ?void* storage = ( >> >> (boost::python::converter::rvalue_from_python_storage*) >> ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; >> >> ? ?wchar_t* w = new wchar_t[l]; >> ? ?for(int i=0;i> ? ? ?w[i] = value[i]; >> ? ?} >> ? ?data->convertible = w; >> ?} >> }; >> ?void init_module() >> ?{ >> ? ?to_python_converter(); >> ? ?wchar_t_from_python_str(); >> ? } >> >> BOOST_PYTHON_MODULE(queryparser){ >> ?init_module(); >> } >> >> But it doesn't seem to be working. It compiles, but python doesn't >> understand the conversion. >> Any thoughts? >> >> >> Thanks in advance! >> >> -- >> Nicolas Lara >> Linux user #380134 >> http://nicolas-lara.blogspot.com/ >> Public key id: 0x152e7713 at http://subkeys.pgp.net/ >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> http://mail.python.org/mailman/listinfo/cplusplus-sig >> > > > > -- > Renato Araujo Oliveira Filho > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Nicolas Lara Linux user #380134 http://nicolas-lara.blogspot.com/ Public key id: 0x152e7713 at http://subkeys.pgp.net/ From renatox at gmail.com Mon Jun 22 20:57:02 2009 From: renatox at gmail.com (Renato Araujo) Date: Mon, 22 Jun 2009 15:57:02 -0300 Subject: [C++-sig] wchar_t* type conversion In-Reply-To: <46ef4bec0906221027l3ed677e0x28162495332fdab@mail.gmail.com> References: <46ef4bec0906210806tb5cd894q7dcd5fdd42eeab8e@mail.gmail.com> <46ef4bec0906220431s4f7899dflca213c20c85c96e1@mail.gmail.com> <95291a80906220648y36ce918amea421d100b8d5549@mail.gmail.com> <46ef4bec0906221027l3ed677e0x28162495332fdab@mail.gmail.com> Message-ID: <95291a80906221157i61a63852o1defb705ccc0bd82@mail.gmail.com> Hi Nicolas At my first look I get this working changing the register function: << to_python_converter(); >> to_python_converter(); << static PyObject* convert(wchar_t* const& w) >> static PyObject* convert(wchar_t const& w) << wchar_t* w2 = w; << while(*w2++ != 0){ << s += *w2; >> wchar_t w2 = w; >> while(w2++ != 0){ >> s += w2; and the function signature: << void testArgument(wchar_t * w){ >> void testArgument(const wchar_t &w){ this make everything works fine: I don't know how get this working with wchar_t* in function signature. Maybe create a class_ for w_char_t. On Mon, Jun 22, 2009 at 2:27 PM, Nicolas Lara wrote: > Hi Renato, > Thanks for the reply. Here's some test code. (I pasted it to make it > more friendly =) > http://paste.pocoo.org/show/124521/ > > Testing it in python: http://paste.pocoo.org/show/124523/ > > Thanks in advance! > Best Regards > > For the archives: > > === test.cpp > > #include "CLucene.h" > #include > #include > > using namespace boost::python; > > struct wchar_t_to_python_str > { > ?static PyObject* convert(wchar_t* const& w) > ?{ > ? ?string s = ""; > ? ?wchar_t* w2 = w; > ? ?while(*w2++ != 0){ > ? ? ?s += *w2; > ? ?} > ? ?return boost::python::incref(boost::python::object(s).ptr()); > ?} > }; > > struct wchar_t_from_python_str > { > ?wchar_t_from_python_str() > ?{ > ? ?boost::python::converter::registry::push_back( > ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, > ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, > ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); > ?} > > ?static void* convertible(PyObject* obj_ptr) > ?{ > ? ?if (!PyString_Check(obj_ptr)) return 0; > ? ?return obj_ptr; > ?} > > ?static void construct( > ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, > > boost::python::converter::rvalue_from_python_stage1_data* data) > ?{ > ? ?const char* value = PyString_AsString(obj_ptr); > ? ?Py_ssize_t l = PyString_Size(obj_ptr); > ? ?if (value == 0) boost::python::throw_error_already_set(); > ? ?void* storage = ( > > (boost::python::converter::rvalue_from_python_storage*) > ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; > > ? ?wchar_t* w = new wchar_t[l]; > ? ?for(int i=0;i ? ? ?w[i] = value[i]; > ? ?} > ? ?data->convertible = w; > ?} > }; > > void init_module() > { > ?to_python_converter(); > ?wchar_t_from_python_str(); > } > > > struct Test { > ?Test(){} > > ?wchar_t * testReturnValue(string s){ > ? ?wchar_t *p=new wchar_t[s.size()]; > ? ?for(string::size_type i=0; i ? ? ?p[i]=s[i]; > ? ?return p; > ?} > > ?void testArgument(wchar_t * w){ > ? ?printf("ok\n"); > ?} > }; > > > BOOST_PYTHON_MODULE(test){ > ?init_module(); > ?class_("Test") > ? ?//.def("testReturnValue", &Test::testReturnValue) // <- desn't compile > ? ?.def("testArgument", &Test::testArgument); > ?; > > } > > ==== testing it in python >>>> import test > >>>> t = test.Test() > >>>> t.testArgument(u"") > --------------------------------------------------------------------------- > ArgumentError ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) > > /home/.../build/ in () > > ArgumentError: Python argument types in > ? ?Test.testArgument(Test, unicode) > did not match C++ signature: > ? ?testArgument(Test {lvalue}, wchar_t*) > >>>> t.testArgument("") > --------------------------------------------------------------------------- > ArgumentError ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) > > /home/.../build/ in () > > ArgumentError: Python argument types in > ? ?Test.testArgument(Test, str) > did not match C++ signature: > ? ?testArgument(Test {lvalue}, wchar_t*) > >>>> from ctypes import c_wchar_p > >>>> c_wchar_p("hello") >>>> c_wchar_p(u'hello') > >>>> w = c_wchar_p("hello") > >>>> t.testArgument(w) > --------------------------------------------------------------------------- > ArgumentError ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) > > /home/nicolas/devel/CyLucene/build/ in () > > ArgumentError: Python argument types in > ? ?Test.testArgument(Test, c_wchar_p) > did not match C++ signature: > ? ?testArgument(Test {lvalue}, wchar_t*) > > > On Mon, Jun 22, 2009 at 1:48 PM, Renato Araujo wrote: >> Hi Nicolas, >> >> Can ?you send a simple example, where the convention fail (c++ to >> python or python to c++)? What kind of test you are running? >> >> BR >> Renato >> >> >> >> On Mon, Jun 22, 2009 at 8:31 AM, Nicolas Lara wrote: >>> Hello, >>> I'm new to Boost::Python and am encountering a problem that seem to be >>> quite common: I need to expose a class to C++ whose constructor takes >>> a wchar_t* as a parameter. >>> The problem is that there is no default type converter from wchar_t to >>> python and vice-versa. >>> >>> I wanted to know if there was a standard way of doing this. Or if >>> anyone could point me to an implementation of this type conversion. >>> >>> So far I'm doing this (based on the faq): >>> >>> struct wchar_t_to_python_str >>> { >>> ?static PyObject* convert(wchar_t* const& w) >>> ?{ >>> ? ?string s = ""; >>> ? ?wchar_t* w2 = w; >>> ? ?while(*w2++ != 0){ >>> ? ? ?s += *w2; >>> ? ?} >>> ? ?return boost::python::incref(boost::python::object(s).ptr()); >>> ?} >>> }; >>> >>> struct wchar_t_from_python_str >>> { >>> ?wchar_t_from_python_str() >>> ?{ >>> ? ?boost::python::converter::registry::push_back( >>> ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, >>> ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, >>> ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); >>> ?} >>> >>> ?static void* convertible(PyObject* obj_ptr) >>> ?{ >>> ? ?if (!PyString_Check(obj_ptr)) return 0; >>> ? ?return obj_ptr; >>> ?} >>> >>> ?static void construct( >>> ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, >>> >>> boost::python::converter::rvalue_from_python_stage1_data* data) >>> ?{ >>> ? ?const char* value = PyString_AsString(obj_ptr); >>> ? ?Py_ssize_t l = PyString_Size(obj_ptr); >>> ? ?if (value == 0) boost::python::throw_error_already_set(); >>> ? ?void* storage = ( >>> >>> (boost::python::converter::rvalue_from_python_storage*) >>> ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; >>> >>> ? ?wchar_t* w = new wchar_t[l]; >>> ? ?for(int i=0;i>> ? ? ?w[i] = value[i]; >>> ? ?} >>> ? ?data->convertible = w; >>> ?} >>> }; >>> ?void init_module() >>> ?{ >>> ? ?to_python_converter(); >>> ? ?wchar_t_from_python_str(); >>> ? } >>> >>> BOOST_PYTHON_MODULE(queryparser){ >>> ?init_module(); >>> } >>> >>> But it doesn't seem to be working. It compiles, but python doesn't >>> understand the conversion. >>> Any thoughts? >>> >>> >>> Thanks in advance! >>> >>> -- >>> Nicolas Lara >>> Linux user #380134 >>> http://nicolas-lara.blogspot.com/ >>> Public key id: 0x152e7713 at http://subkeys.pgp.net/ >>> _______________________________________________ >>> Cplusplus-sig mailing list >>> Cplusplus-sig at python.org >>> http://mail.python.org/mailman/listinfo/cplusplus-sig >>> >> >> >> >> -- >> Renato Araujo Oliveira Filho >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> http://mail.python.org/mailman/listinfo/cplusplus-sig >> > > > > -- > Nicolas Lara > Linux user #380134 > http://nicolas-lara.blogspot.com/ > Public key id: 0x152e7713 at http://subkeys.pgp.net/ > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- BR Renato Araujo Oliveira Filho From nicolaslara at gmail.com Mon Jun 22 21:03:22 2009 From: nicolaslara at gmail.com (Nicolas Lara) Date: Mon, 22 Jun 2009 12:03:22 -0700 Subject: [C++-sig] wchar_t* type conversion In-Reply-To: <95291a80906221157i61a63852o1defb705ccc0bd82@mail.gmail.com> References: <46ef4bec0906210806tb5cd894q7dcd5fdd42eeab8e@mail.gmail.com> <46ef4bec0906220431s4f7899dflca213c20c85c96e1@mail.gmail.com> <95291a80906220648y36ce918amea421d100b8d5549@mail.gmail.com> <46ef4bec0906221027l3ed677e0x28162495332fdab@mail.gmail.com> <95291a80906221157i61a63852o1defb705ccc0bd82@mail.gmail.com> Message-ID: <46ef4bec0906221203v45ea6d9bp777a30dacba65ccb@mail.gmail.com> Thanks! I'm trying to create an interface for a third party library, so I can't actually change it. I also tried using ctypes.pointer but didn't work either. I'll probably need to create a wrapper that does the type conversion. Thanks again Renato! If anyone knows how to make it work with wchar_t* i'd appreciate it! On Mon, Jun 22, 2009 at 11:57 AM, Renato Araujo wrote: > Hi Nicolas > > At my first look I get this working changing the register function: > > << to_python_converter(); >>> to_python_converter(); > > << static PyObject* convert(wchar_t* const& w) >>> static PyObject* convert(wchar_t const& w) > > << ? ?wchar_t* w2 = w; > << ? while(*w2++ != 0){ > << ? ? ?s += *w2; >>> ? wchar_t w2 = w; >>> ? while(w2++ != 0){ >>> ? ? ?s += w2; > > and the function signature: > > << void testArgument(wchar_t * w){ >>> void testArgument(const wchar_t &w){ > > this make everything works fine: > > I don't know how get this working with wchar_t* in function signature. > Maybe create a class_ for w_char_t. > > > > On Mon, Jun 22, 2009 at 2:27 PM, Nicolas Lara wrote: >> Hi Renato, >> Thanks for the reply. Here's some test code. (I pasted it to make it >> more friendly =) >> http://paste.pocoo.org/show/124521/ >> >> Testing it in python: http://paste.pocoo.org/show/124523/ >> >> Thanks in advance! >> Best Regards >> >> For the archives: >> >> === test.cpp >> >> #include "CLucene.h" >> #include >> #include >> >> using namespace boost::python; >> >> struct wchar_t_to_python_str >> { >> ?static PyObject* convert(wchar_t* const& w) >> ?{ >> ? ?string s = ""; >> ? ?wchar_t* w2 = w; >> ? ?while(*w2++ != 0){ >> ? ? ?s += *w2; >> ? ?} >> ? ?return boost::python::incref(boost::python::object(s).ptr()); >> ?} >> }; >> >> struct wchar_t_from_python_str >> { >> ?wchar_t_from_python_str() >> ?{ >> ? ?boost::python::converter::registry::push_back( >> ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, >> ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, >> ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); >> ?} >> >> ?static void* convertible(PyObject* obj_ptr) >> ?{ >> ? ?if (!PyString_Check(obj_ptr)) return 0; >> ? ?return obj_ptr; >> ?} >> >> ?static void construct( >> ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, >> >> boost::python::converter::rvalue_from_python_stage1_data* data) >> ?{ >> ? ?const char* value = PyString_AsString(obj_ptr); >> ? ?Py_ssize_t l = PyString_Size(obj_ptr); >> ? ?if (value == 0) boost::python::throw_error_already_set(); >> ? ?void* storage = ( >> >> (boost::python::converter::rvalue_from_python_storage*) >> ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; >> >> ? ?wchar_t* w = new wchar_t[l]; >> ? ?for(int i=0;i> ? ? ?w[i] = value[i]; >> ? ?} >> ? ?data->convertible = w; >> ?} >> }; >> >> void init_module() >> { >> ?to_python_converter(); >> ?wchar_t_from_python_str(); >> } >> >> >> struct Test { >> ?Test(){} >> >> ?wchar_t * testReturnValue(string s){ >> ? ?wchar_t *p=new wchar_t[s.size()]; >> ? ?for(string::size_type i=0; i> ? ? ?p[i]=s[i]; >> ? ?return p; >> ?} >> >> ?void testArgument(wchar_t * w){ >> ? ?printf("ok\n"); >> ?} >> }; >> >> >> BOOST_PYTHON_MODULE(test){ >> ?init_module(); >> ?class_("Test") >> ? ?//.def("testReturnValue", &Test::testReturnValue) // <- desn't compile >> ? ?.def("testArgument", &Test::testArgument); >> ?; >> >> } >> >> ==== testing it in python >>>>> import test >> >>>>> t = test.Test() >> >>>>> t.testArgument(u"") >> --------------------------------------------------------------------------- >> ArgumentError ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) >> >> /home/.../build/ in () >> >> ArgumentError: Python argument types in >> ? ?Test.testArgument(Test, unicode) >> did not match C++ signature: >> ? ?testArgument(Test {lvalue}, wchar_t*) >> >>>>> t.testArgument("") >> --------------------------------------------------------------------------- >> ArgumentError ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) >> >> /home/.../build/ in () >> >> ArgumentError: Python argument types in >> ? ?Test.testArgument(Test, str) >> did not match C++ signature: >> ? ?testArgument(Test {lvalue}, wchar_t*) >> >>>>> from ctypes import c_wchar_p >> >>>>> c_wchar_p("hello") >>>>> c_wchar_p(u'hello') >> >>>>> w = c_wchar_p("hello") >> >>>>> t.testArgument(w) >> --------------------------------------------------------------------------- >> ArgumentError ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last) >> >> /home/nicolas/devel/CyLucene/build/ in () >> >> ArgumentError: Python argument types in >> ? ?Test.testArgument(Test, c_wchar_p) >> did not match C++ signature: >> ? ?testArgument(Test {lvalue}, wchar_t*) >> >> >> On Mon, Jun 22, 2009 at 1:48 PM, Renato Araujo wrote: >>> Hi Nicolas, >>> >>> Can ?you send a simple example, where the convention fail (c++ to >>> python or python to c++)? What kind of test you are running? >>> >>> BR >>> Renato >>> >>> >>> >>> On Mon, Jun 22, 2009 at 8:31 AM, Nicolas Lara wrote: >>>> Hello, >>>> I'm new to Boost::Python and am encountering a problem that seem to be >>>> quite common: I need to expose a class to C++ whose constructor takes >>>> a wchar_t* as a parameter. >>>> The problem is that there is no default type converter from wchar_t to >>>> python and vice-versa. >>>> >>>> I wanted to know if there was a standard way of doing this. Or if >>>> anyone could point me to an implementation of this type conversion. >>>> >>>> So far I'm doing this (based on the faq): >>>> >>>> struct wchar_t_to_python_str >>>> { >>>> ?static PyObject* convert(wchar_t* const& w) >>>> ?{ >>>> ? ?string s = ""; >>>> ? ?wchar_t* w2 = w; >>>> ? ?while(*w2++ != 0){ >>>> ? ? ?s += *w2; >>>> ? ?} >>>> ? ?return boost::python::incref(boost::python::object(s).ptr()); >>>> ?} >>>> }; >>>> >>>> struct wchar_t_from_python_str >>>> { >>>> ?wchar_t_from_python_str() >>>> ?{ >>>> ? ?boost::python::converter::registry::push_back( >>>> ? ? ? ? ? ? ? ? ? ? ? ? ? &convertible, >>>> ? ? ? ? ? ? ? ? ? ? ? ? ? &construct, >>>> ? ? ? ? ? ? ? ? ? ? ? ? ? boost::python::type_id()); >>>> ?} >>>> >>>> ?static void* convertible(PyObject* obj_ptr) >>>> ?{ >>>> ? ?if (!PyString_Check(obj_ptr)) return 0; >>>> ? ?return obj_ptr; >>>> ?} >>>> >>>> ?static void construct( >>>> ? ? ? ? ? ? ? ? ? ? ? ?PyObject* obj_ptr, >>>> >>>> boost::python::converter::rvalue_from_python_stage1_data* data) >>>> ?{ >>>> ? ?const char* value = PyString_AsString(obj_ptr); >>>> ? ?Py_ssize_t l = PyString_Size(obj_ptr); >>>> ? ?if (value == 0) boost::python::throw_error_already_set(); >>>> ? ?void* storage = ( >>>> >>>> (boost::python::converter::rvalue_from_python_storage*) >>>> ? ? ? ? ? ? ? ? ? ? data)->storage.bytes; >>>> >>>> ? ?wchar_t* w = new wchar_t[l]; >>>> ? ?for(int i=0;i>>> ? ? ?w[i] = value[i]; >>>> ? ?} >>>> ? ?data->convertible = w; >>>> ?} >>>> }; >>>> ?void init_module() >>>> ?{ >>>> ? ?to_python_converter(); >>>> ? ?wchar_t_from_python_str(); >>>> ? } >>>> >>>> BOOST_PYTHON_MODULE(queryparser){ >>>> ?init_module(); >>>> } >>>> >>>> But it doesn't seem to be working. It compiles, but python doesn't >>>> understand the conversion. >>>> Any thoughts? >>>> >>>> >>>> Thanks in advance! >>>> >>>> -- >>>> Nicolas Lara >>>> Linux user #380134 >>>> http://nicolas-lara.blogspot.com/ >>>> Public key id: 0x152e7713 at http://subkeys.pgp.net/ >>>> _______________________________________________ >>>> Cplusplus-sig mailing list >>>> Cplusplus-sig at python.org >>>> http://mail.python.org/mailman/listinfo/cplusplus-sig >>>> >>> >>> >>> >>> -- >>> Renato Araujo Oliveira Filho >>> _______________________________________________ >>> Cplusplus-sig mailing list >>> Cplusplus-sig at python.org >>> http://mail.python.org/mailman/listinfo/cplusplus-sig >>> >> >> >> >> -- >> Nicolas Lara >> Linux user #380134 >> http://nicolas-lara.blogspot.com/ >> Public key id: 0x152e7713 at http://subkeys.pgp.net/ >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> http://mail.python.org/mailman/listinfo/cplusplus-sig >> > > > > -- > BR > Renato Araujo Oliveira Filho > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Nicolas Lara Linux user #380134 http://nicolas-lara.blogspot.com/ Public key id: 0x152e7713 at http://subkeys.pgp.net/ From mikeowen at llnl.gov Mon Jun 22 23:31:13 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Mon, 22 Jun 2009 14:31:13 -0700 Subject: [C++-sig] pybindgen: Why does pybindgen want to call constructors for a singleton class? Message-ID: <2C3E4E08-541B-46FC-B57C-1BB675F22D63@llnl.gov> I'm looking at wrapping a C++ singleton with pybindgen, and it seems that if I expose the method for getting the instance to python the generated code wants to call a copy constructor, which seems wrong to me. If for instance I define a class "A" as a singleton: class A { public: static A* instancePtr() { if (A::mInstancePtr == 0) A::mInstancePtr = new A; return mInstancePtr; } private: A(); A(const A& rhs); A& operator=(const A& rhs); static A* mInstancePtr; }; and I wrap A and its instance method like so: mod = Module("singleton_example") mod.add_include('"singleton_example.hh"') x = mod.add_class("A", is_singleton=True) x.add_method("instancePtr", retval("A*", caller_owns_return=False), []) pybindgen generates the following code for the instancePtr method: PyObject * _wrap_PyA_instancePtr(PyA *self) { PyObject *py_retval; A *retval; PyA *py_A; retval = self->obj->instancePtr(); if (!(retval)) { Py_INCREF(Py_None); return Py_None; } py_A = PyObject_New(PyA, &PyA_Type); py_A->obj = new A(*retval); py_retval = Py_BuildValue((char *) "N", py_A); return py_retval; } As you can see it is trying to construct a new object with a copy of the A retval parameter (the line that reads "py_A->obj = new A(*retval);". This is of course forbidden because all of A's constructors are private -- as a singleton we don't want anyone calling constructors on this object. Moreover, since I exposed "instancePtr" as returning a pointer I did not expect any copies to be generated anyway -- I'd like to see the pointer "retval" sent back directly (even if this wasn't a singleton). Am I missing some syntax here to prevent pybindgen from trying to make these copies? Thanks! Mike. From gjcarneiro at gmail.com Tue Jun 23 02:11:14 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 23 Jun 2009 01:11:14 +0100 Subject: [C++-sig] pybindgen: Why does pybindgen want to call constructors for a singleton class? In-Reply-To: <2C3E4E08-541B-46FC-B57C-1BB675F22D63@llnl.gov> References: <2C3E4E08-541B-46FC-B57C-1BB675F22D63@llnl.gov> Message-ID: 2009/6/22 J. Michael Owen > I'm looking at wrapping a C++ singleton with pybindgen, and it seems that > if I expose the method for getting the instance to python the generated code > wants to call a copy constructor, which seems wrong to me. If for instance > I define a class "A" as a singleton: > > class A { > public: > static A* instancePtr() { > if (A::mInstancePtr == 0) A::mInstancePtr = new A; > return mInstancePtr; > } > private: > A(); > A(const A& rhs); > A& operator=(const A& rhs); > static A* mInstancePtr; > }; > > and I wrap A and its instance method like so: > > mod = Module("singleton_example") > mod.add_include('"singleton_example.hh"') > x = mod.add_class("A", is_singleton=True) > x.add_method("instancePtr", retval("A*", caller_owns_return=False), []) > > pybindgen generates the following code for the instancePtr method: > > PyObject * > _wrap_PyA_instancePtr(PyA *self) > { > PyObject *py_retval; > A *retval; > PyA *py_A; > > retval = self->obj->instancePtr(); > if (!(retval)) { > Py_INCREF(Py_None); > return Py_None; > } > py_A = PyObject_New(PyA, &PyA_Type); > py_A->obj = new A(*retval); > py_retval = Py_BuildValue((char *) "N", py_A); > return py_retval; > } > > As you can see it is trying to construct a new object with a copy of the A > retval parameter (the line that reads "py_A->obj = new A(*retval);". This > is of course forbidden because all of A's constructors are private -- as a > singleton we don't want anyone calling constructors on this object. > Moreover, since I exposed "instancePtr" as returning a pointer I did not > expect any copies to be generated anyway -- I'd like to see the pointer > "retval" sent back directly (even if this wasn't a singleton). Am I missing > some syntax here to prevent pybindgen from trying to make these copies? It tries to copy because you say caller_owns_return=False, pybindgen needs to copy the object so that the wrapper can own the object. What you can do here is use caller_owns_return=True instead of False. It's OK in this case; because of the is_singleton option, the C++ wrapped object will never be freed when the wrapper is destroyed. -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeowen at llnl.gov Tue Jun 23 02:23:33 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Mon, 22 Jun 2009 17:23:33 -0700 Subject: [C++-sig] pybindgen: Why does pybindgen want to call constructors for a singleton class? In-Reply-To: References: <2C3E4E08-541B-46FC-B57C-1BB675F22D63@llnl.gov> Message-ID: <82BBF051-522B-438A-85F2-C5313CB58FD8@llnl.gov> Ah, I see! That does the trick! Thank you very much. I was clearly confused about what "caller_owns_return" meant -- I had it backwards. Mike. On Jun 22, 2009, at 5:11 PM, Gustavo Carneiro wrote: > > > 2009/6/22 J. Michael Owen > I'm looking at wrapping a C++ singleton with pybindgen, and it seems > that if I expose the method for getting the instance to python the > generated code wants to call a copy constructor, which seems wrong > to me. If for instance I define a class "A" as a singleton: > > class A { > public: > static A* instancePtr() { > if (A::mInstancePtr == 0) A::mInstancePtr = new A; > return mInstancePtr; > } > private: > A(); > A(const A& rhs); > A& operator=(const A& rhs); > static A* mInstancePtr; > }; > > and I wrap A and its instance method like so: > > mod = Module("singleton_example") > mod.add_include('"singleton_example.hh"') > x = mod.add_class("A", is_singleton=True) > x.add_method("instancePtr", retval("A*", caller_owns_return=False), > []) > > pybindgen generates the following code for the instancePtr method: > > PyObject * > _wrap_PyA_instancePtr(PyA *self) > { > PyObject *py_retval; > A *retval; > PyA *py_A; > > retval = self->obj->instancePtr(); > if (!(retval)) { > Py_INCREF(Py_None); > return Py_None; > } > py_A = PyObject_New(PyA, &PyA_Type); > py_A->obj = new A(*retval); > py_retval = Py_BuildValue((char *) "N", py_A); > return py_retval; > } > > As you can see it is trying to construct a new object with a copy of > the A retval parameter (the line that reads "py_A->obj = new > A(*retval);". This is of course forbidden because all of A's > constructors are private -- as a singleton we don't want anyone > calling constructors on this object. Moreover, since I exposed > "instancePtr" as returning a pointer I did not expect any copies to > be generated anyway -- I'd like to see the pointer "retval" sent > back directly (even if this wasn't a singleton). Am I missing some > syntax here to prevent pybindgen from trying to make these copies? > > It tries to copy because you say caller_owns_return=False, pybindgen > needs to copy the object so that the wrapper can own the object. > What you can do here is use caller_owns_return=True instead of > False. It's OK in this case; because of the is_singleton option, > the C++ wrapped object will never be freed when the wrapper is > destroyed. > > -- > Gustavo J. A. M. Carneiro > INESC Porto, Telecommunications and Multimedia Unit > "The universe is always one step beyond logic." -- Frank Herbert > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http:// mail.python.org/mailman/listinfo/cplusplus-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From rwgk at yahoo.com Tue Jun 23 11:10:10 2009 From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve) Date: Tue, 23 Jun 2009 02:10:10 -0700 (PDT) Subject: [C++-sig] boost.python: no overflow checking for unsigned int parameters ? In-Reply-To: <55b616790906180608r7b7282b5s750a5cff625a5f4@mail.gmail.com> References: <55b616790906151224g6ae9a712ke3dab8dfe3e1c556@mail.gmail.com> <59161.36991.qm@web111401.mail.gq1.yahoo.com> <55b616790906180608r7b7282b5s750a5cff625a5f4@mail.gmail.com> Message-ID: <515613.42813.qm@web111416.mail.gq1.yahoo.com> Thanks for your work! I'm out of town, back in a couple weeks. I'll integrate your changes when I'm back (unless anyone else gets to it before I'm back). Ralf ----- Original Message ---- From: Anderson Lizardo To: Development of Python/C++ integration Sent: Thursday, June 18, 2009 6:08:20 AM Subject: Re: [C++-sig] boost.python: no overflow checking for unsigned int parameters ? On Mon, Jun 15, 2009 at 4:23 PM, Ralf W. Grosse-Kunstleve wrote: > > If you send me a patch (against the boost trunk) I'll test it and check it in. > Probably, the file to be changed is > boost/libs/python/src/converter/builtin_converters.cpp > I'd try working with > boost/numeric/conversion/cast.hpp Done. See https://svn.boost.org/trac/boost/ticket/3189 and the patch attached to it. Note the patch still lacks the addition of a test to the test suite. I'm working on it, but you can test it in advance. Regards, -- Anderson Lizardo OpenBossa Labs - INdT Manaus - Brazil From bfitzpatrick at vtiinstruments.com Tue Jun 23 17:09:18 2009 From: bfitzpatrick at vtiinstruments.com (Ben Fitzpatrick) Date: Tue, 23 Jun 2009 11:09:18 -0400 Subject: [C++-sig] Pybindgen/pygccxml integration Message-ID: <4A40F01E.3030803@vtiinstruments.com> Hi everyone, I'm just starting to check out pybindgen after the messages I've seen floating around on this mailing list, and I have a question about the pygccxml integration. I've made a pure-virtual class with a pointer return type, like so: class pure_virtual_class { virtual int get_value(int* value)=0; virtual int put_value(int value)=0; static pure_virtual_class* Create(); }; Normally I'd call add_function('get_value', retval(int), [param('int *', 'value', transfer_ownership=True)]) (I think) But since I'm scanning the headers with gccxml, I'm not sure how to tell it to transfer the ownership. Right now it's giving me a TypeConfigurationError during 'parse', which appears to be because it doesn't know how to deal with the ownership. I looked in the documentation and couldn't find anything referencing this, and none of the examples use gccxml. Any ideas? Thanks, Ben Fitzpatrick From gjcarneiro at gmail.com Tue Jun 23 17:39:09 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 23 Jun 2009 16:39:09 +0100 Subject: [C++-sig] Pybindgen/pygccxml integration In-Reply-To: <4A40F01E.3030803@vtiinstruments.com> References: <4A40F01E.3030803@vtiinstruments.com> Message-ID: 2009/6/23 Ben Fitzpatrick > Hi everyone, > > I'm just starting to check out pybindgen after the messages I've seen > floating around on this mailing list, and I have a question about the > pygccxml integration. I've made a pure-virtual class with a pointer return > type, like so: > > class pure_virtual_class > { > virtual int get_value(int* value)=0; > virtual int put_value(int value)=0; > > static pure_virtual_class* Create(); > }; > > Normally I'd call add_function('get_value', retval(int), [param('int *', > 'value', transfer_ownership=True)]) (I think) > But since I'm scanning the headers with gccxml, I'm not sure how to tell it > to transfer the ownership. > Right now it's giving me a TypeConfigurationError during 'parse', which > appears to be because it doesn't know how to deal with the ownership. > > I looked in the documentation and couldn't find anything referencing this, > and none of the examples use gccxml. > > Any ideas? Yes, there are two possible (alternative) ways to deal with this: 1. Add inline annotations as C++ comments in the header file to be scanned; 2. Add scanning hooks to inject annotations via a function callback; For inline annotations, see tests/foo.h, look for the comment lines containing -#-. I must warn you, though, that annotations are kind of a "brittle" system. The annotations are searched upwards starting in the line immediately before the line that gccxml reports as the line the declaration is defined. The problem here is that gccxml reports the line of a declaration sometimes in weird and unexpected places. The other way is more complicated in terms of code, but doable with some pacience. You can see an example in ns-3 [1], in the pre_scan_hook function. [1] http://code.nsnam.org/ns-3-dev/file/96811f76c3e2/bindings/python/ns3modulescan.py I hope this helps. -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From bfitzpatrick at vtiinstruments.com Tue Jun 23 18:43:55 2009 From: bfitzpatrick at vtiinstruments.com (Ben Fitzpatrick) Date: Tue, 23 Jun 2009 12:43:55 -0400 Subject: [C++-sig] Pybindgen/pygccxml integration In-Reply-To: References: <4A40F01E.3030803@vtiinstruments.com> Message-ID: <4A41064B.7090300@vtiinstruments.com> Thanks for the suggestions! I tried the first one, just as a test. I'd like to do the second programatically if I can, but I just wanted to make sure annotations were going to fix this. It still seems to be giving me the same error: (...)/pure_virtual.h:5: WrapperWarning: Parameter 'int * value' error (used in int pure_virtual_class::get_value(int * value) [member function]): virtual int get_value(int* value) = 0; Here is what I changed my code to: // -#- @value(transfer_ownership=true) -#- virtual int get_value(int* value)=0; This appears to be similar to the declaration you pointed me to in foo.h: // -#- @zbr(transfer_ownership=true) -#- void store_zbr (Zbr *zbr); Also tried renaming it to 'foobaz' in case 'value' was a keyword. Is there something I'm missing? Thanks, Ben Fitzpatrick Gustavo Carneiro wrote: > > > 2009/6/23 Ben Fitzpatrick > > > Hi everyone, > > I'm just starting to check out pybindgen after the messages I've > seen floating around on this mailing list, and I have a question > about the pygccxml integration. I've made a pure-virtual class > with a pointer return type, like so: > > class pure_virtual_class > { > virtual int get_value(int* value)=0; > virtual int put_value(int value)=0; > > static pure_virtual_class* Create(); > }; > > Normally I'd call add_function('get_value', retval(int), > [param('int *', 'value', transfer_ownership=True)]) (I think) > But since I'm scanning the headers with gccxml, I'm not sure how > to tell it to transfer the ownership. > Right now it's giving me a TypeConfigurationError during 'parse', > which appears to be because it doesn't know how to deal with the > ownership. > > I looked in the documentation and couldn't find anything > referencing this, and none of the examples use gccxml. > > Any ideas? > > > Yes, there are two possible (alternative) ways to deal with this: > > 1. Add inline annotations as C++ comments in the header file to be > scanned; > 2. Add scanning hooks to inject annotations via a function callback; > > For inline annotations, see tests/foo.h, look for the comment lines > containing -#-. I must warn you, though, that annotations are kind of > a "brittle" system. The annotations are searched upwards starting in > the line immediately before the line that gccxml reports as the line > the declaration is defined. The problem here is that gccxml reports > the line of a declaration sometimes in weird and unexpected places. > > The other way is more complicated in terms of code, but doable with > some pacience. You can see an example in ns-3 [1], in the > pre_scan_hook function. > > > [1] > http://code.nsnam.org/ns-3-dev/file/96811f76c3e2/bindings/python/ns3modulescan.py > > I hope this helps. > > -- > Gustavo J. A. M. Carneiro > INESC Porto, Telecommunications and Multimedia Unit > "The universe is always one step beyond logic." -- Frank Herbert From sipickles at googlemail.com Tue Jun 23 20:05:11 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Tue, 23 Jun 2009 19:05:11 +0100 Subject: [C++-sig] bump.... boost::python and __declspec(align(16)) Message-ID: <4A411957.8090409@googlemail.com> Hello all, Sorry to bump this, but i am still unable to resolve my problem described below. I have seen a 2008 thread on this: http://archives.free.net.ph/message/20080331.203857.548691a0.ja.html In this case it is align(8) not align(16), and the thread describes this issue as resolved. This leads me to believe there is an answer *out there* :) Thanks for your help Simon Hi, I think this is an MSVC problem caused by the bad STL implementation, but I wondered if there was a work around. Both my client (Win32 MSVC8) and server (ubuntu GCC 4.2.3) share the same c++ code. It compiles and runs on GCC but fails on MSVC. This uses the the bullet physics library and includes a type defined like this: __declspec(align(16)) btVector3 {/*...*/}; When building my boost::python extension with: class_("BtVector3", init()); I get an error like this : C:\boost_1_38_0\boost/python/converter/as_to_python_function.hpp(21) : error C2719: 'unnamed-parameter': formal parameter with __declspec(align('16')) won't be aligned C:\boost_1_38_0\boost/python/to_python_converter.hpp(88) : see reference to class template instantiation 'boost::python::converter::as_to_python_function' being compiled with [ T=btVector3, /* .... Huge call stack omitted for clarity.... */ build.cpp(155) : see reference to class template instantiation 'boost::python::class_' being compiled with [ W=btVector3 ] Is there a way to overcome this? switch to MinGW? :) Thank you very much Simon From seefeld at sympatico.ca Tue Jun 23 20:14:10 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue, 23 Jun 2009 14:14:10 -0400 Subject: [C++-sig] bump.... boost::python and __declspec(align(16)) In-Reply-To: <4A411957.8090409@googlemail.com> References: <4A411957.8090409@googlemail.com> Message-ID: <4A411B72.9000305@sympatico.ca> On 06/23/2009 02:05 PM, Simon Pickles wrote: > > Hello all, > > Sorry to bump this, but i am still unable to resolve my problem > described below. The problem doesn't appear to be related to boost.python, or even boost. You may have better luck asking on a more suitable forum. Stefan -- ...ich hab' noch einen Koffer in Berlin... From sipickles at googlemail.com Tue Jun 23 20:18:33 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Tue, 23 Jun 2009 19:18:33 +0100 Subject: [C++-sig] bump.... boost::python and __declspec(align(16)) In-Reply-To: <4A411B72.9000305@sympatico.ca> References: <4A411957.8090409@googlemail.com> <4A411B72.9000305@sympatico.ca> Message-ID: <4A411C79.80909@googlemail.com> Stefan Seefeld wrote: > On 06/23/2009 02:05 PM, Simon Pickles wrote: >> >> Hello all, >> >> Sorry to bump this, but i am still unable to resolve my problem >> described below. > > The problem doesn't appear to be related to boost.python, or even boost. > You may have better luck asking on a more suitable forum. > > Stefan > > I thought someone might say that. Maybe when more experienced, I will be able to ascertain that myself. Thanks Stefan. Simon From cschramm at shakaweb.org Tue Jun 23 20:32:41 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Tue, 23 Jun 2009 20:32:41 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A3E1C17.7070502@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> Message-ID: <4A411FC9.2080004@shakaweb.org> Mmmkay, looks like I found a dirty little workaround: I put the C++ functions, I want to expose into a dummy class: class Dummy { int f1(str arg1, ... void f2(tuple arg1, ... } And then create a bpy object from it: object tmp = class_("dummy") .def("f1", &Dummy::f1) .def("f2", &Dummy::f2); Since I don't want to have this class in Python, I now add the methods to the namespace instead: global["my_mod"].attr("__dict__")["f1"] = tmp.attr("__dict__")["f1"]; global["my_mod"].attr("__dict__")["f2"] = tmp.attr("__dict__")["f2"]; Doing so, I end up with callable functions f1 and f2 in my_mod's namespace which act as if they where simple functions outside of any class, but benefit from boost's functionalities. The only undesirable side effect I spotted is tracebacks exposing the dummy class name: Boost.Python.ArgumentError: Python argument types in dummy.f1(str, str) did not match C++ signature: Message(boost::python::str, boost::python::str, boost::python::str) Note to anyone who's about to reproduce my solution: The dummy class mustn't have a staticmethod("f1") call, otherwise you end up with a non-callable "staticmethod" object f1 in namespace. Any comments welcome. From seefeld at sympatico.ca Tue Jun 23 20:43:31 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue, 23 Jun 2009 14:43:31 -0400 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A411FC9.2080004@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> Message-ID: <4A412253.7010308@sympatico.ca> On 06/23/2009 02:32 PM, Christopher Schramm wrote: > Mmmkay, looks like I found a dirty little workaround: > I read your original mail, but I didn't understand what you are trying to achieve. You certainly can export functions to python: void foo(...); ... bpl::def("foo", foo); works just fine. There shouldn't be any need to wrap the function in a class (member function), and then create some hidden object to invoke it. Or am I misunderstanding what you want ? > global["my_mod"].attr("__dict__")["f1"] = tmp.attr("__dict__")["f1"]; > Why not simply global["my_mod"]["f1"] = ... ; ? HTH, Stefan -- ...ich hab' noch einen Koffer in Berlin... From jimtreadway at gmail.com Tue Jun 23 20:45:11 2009 From: jimtreadway at gmail.com (Jim Treadway) Date: Tue, 23 Jun 2009 11:45:11 -0700 Subject: [C++-sig] boost::python on Linux Message-ID: I'm having trouble getting a simple boost::python sample program to work properly on Linux. It compiles and seems to link properly, but the Python interpreter is unable to call my C++ function. On Mac OS X the program works as expected. Any help would be appreciated, hopefully I'm missing something obvious. The program output is: -- BEGIN -- Traceback (most recent call last): File "", line 1, in Boost.Python.ArgumentError: Python argument types in Test.test(int) did not match C++ signature: test(int) -- END -- Here is the code. It's compiled and linked using "g++ -I/usr/include/python2.5 -lpython2.5 -lboost_python -Wall -o test test.cpp". On the Linux box I'm using boost-1.37.0 and python-2.5.2. -- BEGIN -- #include #include #include using namespace boost::python; /* g++ -I/usr/include/python2.5 -lpython2.5 -lboost_python -Wall -o test test.cpp */ int test(int i) { fprintf(stderr, "%s:\n", __FUNCTION__); return i * 5; } BOOST_PYTHON_MODULE(Test) { using namespace boost::python; def("test", test); } int main(int argc, char *argv[]) { Py_Initialize(); try { initTest(); PyRun_SimpleString("import Test"); PyRun_SimpleString("print 'calling test function'"); PyRun_SimpleString("print Test.test(5)"); } catch (boost::python::error_already_set) { PyErr_Print(); } Py_Finalize(); return 0; } -- END -- From sipickles at googlemail.com Tue Jun 23 20:59:02 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Tue, 23 Jun 2009 19:59:02 +0100 Subject: [C++-sig] Passing a c++ new'ed object back to python Message-ID: <4A4125F6.9070105@googlemail.com> Hello again. Here's the setup #python from hybrid import ObjectManager om = ObjectManager() om.PassModel() //cpp class Model { ~Model() { printf("model go boom"); } /* */ }; class ObjectManager { Model* PassModel() { return new Model(); }; }; When I use code which parallels this structure, I see the Model destructor is called when the c++ object goes out of scope (fine) despite the object being passed to python as the argument of a function (not fine!) Is there a return policy I need to provide? I am confused by the way the destructing class is new'ed in c++ code, not in python by using exposed c++. Better luck this time :) Simon From gjcarneiro at gmail.com Tue Jun 23 20:56:18 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 23 Jun 2009 19:56:18 +0100 Subject: [C++-sig] Pybindgen/pygccxml integration In-Reply-To: <4A41064B.7090300@vtiinstruments.com> References: <4A40F01E.3030803@vtiinstruments.com> <4A41064B.7090300@vtiinstruments.com> Message-ID: 2009/6/23 Ben Fitzpatrick > Thanks for the suggestions! > > I tried the first one, just as a test. I'd like to do the second > programatically if I can, but I just wanted to make sure annotations were > going to fix this. It still seems to be giving me the same error: > (...)/pure_virtual.h:5: WrapperWarning: Parameter 'int * value' error (used > in int pure_virtual_class::get_value(int * value) [member function]): > > virtual int get_value(int* value) = 0; > > Here is what I changed my code to: > > // -#- @value(transfer_ownership=true) -#- > virtual int get_value(int* value)=0; > Apart from the blank line, seems OK. Do you get a "unused annotation" warning? > > This appears to be similar to the declaration you pointed me to in foo.h: > // -#- @zbr(transfer_ownership=true) -#- > void store_zbr (Zbr *zbr); > > Also tried renaming it to 'foobaz' in case 'value' was a keyword. Is there > something I'm missing? > > Thanks, > Ben Fitzpatrick > > > Gustavo Carneiro wrote: > >> >> >> 2009/6/23 Ben Fitzpatrick > bfitzpatrick at vtiinstruments.com>> >> >> >> Hi everyone, >> >> I'm just starting to check out pybindgen after the messages I've >> seen floating around on this mailing list, and I have a question >> about the pygccxml integration. I've made a pure-virtual class >> with a pointer return type, like so: >> >> class pure_virtual_class >> { >> virtual int get_value(int* value)=0; >> virtual int put_value(int value)=0; >> >> static pure_virtual_class* Create(); >> }; >> >> Normally I'd call add_function('get_value', retval(int), >> [param('int *', 'value', transfer_ownership=True)]) (I think) >> But since I'm scanning the headers with gccxml, I'm not sure how >> to tell it to transfer the ownership. >> Right now it's giving me a TypeConfigurationError during 'parse', >> which appears to be because it doesn't know how to deal with the >> ownership. >> >> I looked in the documentation and couldn't find anything >> referencing this, and none of the examples use gccxml. >> >> Any ideas? >> >> >> Yes, there are two possible (alternative) ways to deal with this: >> >> 1. Add inline annotations as C++ comments in the header file to be >> scanned; >> 2. Add scanning hooks to inject annotations via a function callback; >> >> For inline annotations, see tests/foo.h, look for the comment lines >> containing -#-. I must warn you, though, that annotations are kind of a >> "brittle" system. The annotations are searched upwards starting in the line >> immediately before the line that gccxml reports as the line the declaration >> is defined. The problem here is that gccxml reports the line of a >> declaration sometimes in weird and unexpected places. >> >> The other way is more complicated in terms of code, but doable with some >> pacience. You can see an example in ns-3 [1], in the pre_scan_hook >> function. >> >> >> [1] >> http://code.nsnam.org/ns-3-dev/file/96811f76c3e2/bindings/python/ns3modulescan.py >> >> I hope this helps. >> >> -- >> Gustavo J. A. M. Carneiro >> INESC Porto, Telecommunications and Multimedia Unit >> "The universe is always one step beyond logic." -- Frank Herbert >> > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From seefeld at sympatico.ca Tue Jun 23 21:06:00 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue, 23 Jun 2009 15:06:00 -0400 Subject: [C++-sig] boost::python on Linux In-Reply-To: References: Message-ID: <4A412798.4040901@sympatico.ca> On 06/23/2009 02:45 PM, Jim Treadway wrote: > I'm having trouble getting a simple boost::python sample program to > work properly on Linux. It compiles and seems to link properly, but the Python > interpreter is unable to call my C++ function. On Mac OS X the > program works as expected. > > Any help would be appreciated, hopefully I'm missing something obvious. > Let's see: I expect the problem to be that the Python interpreter doesn't 'see' the 'Test' module. While normally this would just result in an import error, here it may just happen to find a different 'Test' module, which, as it doesn't match what you expect, raises an ArgumentError. > -- BEGIN -- > #include > #include > #include > > using namespace boost::python; > > /* g++ -I/usr/include/python2.5 -lpython2.5 -lboost_python -Wall -o > test test.cpp */ > > int test(int i) > { > fprintf(stderr, "%s:\n", __FUNCTION__); > return i * 5; > } > > BOOST_PYTHON_MODULE(Test) > { > using namespace boost::python; > def("test", test); > } > > int main(int argc, char *argv[]) > { > Py_Initialize(); > > try { > initTest(); > > I'm actually not sure what the effect of calling 'initTest()' is. I believe you should call: PyImport_AppendInittab(const_cast("Test"), initTest); > PyRun_SimpleString("import Test"); > PyRun_SimpleString("print 'calling test function'"); > PyRun_SimpleString("print Test.test(5)"); > } catch (boost::python::error_already_set) { > PyErr_Print(); > } > > Py_Finalize(); > return 0; > } > -- END -- > HTH, Stefan -- ...ich hab' noch einen Koffer in Berlin... From seefeld at sympatico.ca Tue Jun 23 21:09:35 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue, 23 Jun 2009 15:09:35 -0400 Subject: [C++-sig] Passing a c++ new'ed object back to python In-Reply-To: <4A4125F6.9070105@googlemail.com> References: <4A4125F6.9070105@googlemail.com> Message-ID: <4A41286F.9060302@sympatico.ca> On 06/23/2009 02:59 PM, Simon Pickles wrote: > > When I use code which parallels this structure, I see the Model > destructor is called when the c++ object goes out of scope (fine) > despite the object being passed to python as the argument of a > function (not fine!) That is because by default arguments are passed by-value. If you want something else, you need to specify a return-value policy, such as manage_new_object. > > Is there a return policy I need to provide? I am confused by the way > the destructing class is new'ed in c++ code, not in python by using > exposed c++. boost.python wants to pass a Model, and thus makes a copy. from the object you pass back, then destructs it. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From bfitzpatrick at vtiinstruments.com Tue Jun 23 21:10:13 2009 From: bfitzpatrick at vtiinstruments.com (Ben Fitzpatrick) Date: Tue, 23 Jun 2009 15:10:13 -0400 Subject: [C++-sig] Pybindgen/pygccxml integration In-Reply-To: References: <4A40F01E.3030803@vtiinstruments.com> <4A41064B.7090300@vtiinstruments.com> Message-ID: <4A412895.90206@vtiinstruments.com> Gustavo Carneiro wrote: > > > 2009/6/23 Ben Fitzpatrick > > > Thanks for the suggestions! > > I tried the first one, just as a test. I'd like to do the second > programatically if I can, but I just wanted to make sure > annotations were going to fix this. It still seems to be giving me > the same error: > (...)/pure_virtual.h:5: WrapperWarning: Parameter 'int * value' > error (used in int pure_virtual_class::get_value(int * value) > [member function]): > 0xb780326c> > > virtual int get_value(int* value) = 0; > > Here is what I changed my code to: > > // -#- @value(transfer_ownership=true) -#- > > virtual int get_value(int* value)=0; > > > Apart from the blank line, seems OK. Do you get a "unused annotation" > warning? > That's odd, there wasn't a blank line in my original email. There certainly isn't one in the file. I do not get an unused annotation warning, or anything to suggest it noticed my annotation in the slightest. I am using Python2.4 on Debian Etch, with Pygccxml 0.9.5. The Python's a bit old, but everything else has been compiled or installed manually. From jimtreadway at gmail.com Tue Jun 23 21:17:19 2009 From: jimtreadway at gmail.com (Jim Treadway) Date: Tue, 23 Jun 2009 12:17:19 -0700 Subject: [C++-sig] boost::python on Linux In-Reply-To: <4A412798.4040901@sympatico.ca> References: <4A412798.4040901@sympatico.ca> Message-ID: On Tue, Jun 23, 2009 at 12:06 PM, Stefan Seefeld wrote: > On 06/23/2009 02:45 PM, Jim Treadway wrote: > >> I'm having trouble getting a simple boost::python sample program to >> work properly on Linux. It compiles and seems to link properly, but the >> Python >> interpreter is unable to call my C++ function. On Mac OS X the >> program works as expected. >> >> Any help would be appreciated, hopefully I'm missing something obvious. >> >> > > > Let's see: > > I expect the problem to be that the Python interpreter doesn't 'see' the > 'Test' module. While normally this would just result in an import error, > here it may just happen to find a different 'Test' module, which, as it > doesn't match what you expect, raises an ArgumentError. > Renaming everything from test to "MyUniqueTest" produces the same effective results. -- BEGIN -- > #include > #include > #include > > using namespace boost::python; > > /* g++ -I/usr/include/python2.5 -lpython2.5 -lboost_python -Wall -o > test test.cpp */ > > int test(int i) > { > fprintf(stderr, "%s:\n", __FUNCTION__); > return i * 5; > } > > BOOST_PYTHON_MODULE(Test) > { > using namespace boost::python; > def("test", test); > } > > int main(int argc, char *argv[]) > { > Py_Initialize(); > > try { > initTest(); > I'm actually not sure what the effect of calling 'initTest()' is. > I believe you should call: > > PyImport_AppendInittab(const_cast("Test"), initTest); My understanding is that PyImport_AppendInittab makes it so that a Python "import" statement will call your initialization function so that you do not have to call "initTest" (for example) from the C++ code. Nevertheless, using PyImport_AppendInittab instead, in both the location you recommend as well as before the call to Py_Initialize() produces no change in the results. PyRun_SimpleString("import Test"); > PyRun_SimpleString("print 'calling test function'"); > PyRun_SimpleString("print Test.test(5)"); > } catch (boost::python::error_already_set) { > PyErr_Print(); > } > > Py_Finalize(); > return 0; > } > -- END -- I've also tried: PyRun_SimpleString("import sys, dl"); PyRun_SimpleString("sys.setdlopenflags(dl.RTLD_NOW | dl.RTLD_GLOBAL); before the "import Test" part, but that doesn't change the result either. Jim > -------------- next part -------------- An HTML attachment was scrubbed... URL: From cschramm at shakaweb.org Tue Jun 23 22:03:16 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Tue, 23 Jun 2009 22:03:16 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A412253.7010308@sympatico.ca> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> Message-ID: <4A413504.3080803@shakaweb.org> Stefan Seefeld wrote: > I read your original mail, but I didn't understand what you are trying > to achieve. You certainly can export functions to python: > > void foo(...); > > ... > > bpl::def("foo", foo); > > > works just fine. There shouldn't be any need to wrap the function in a > class (member function), and then create some hidden object to invoke > it. Or am I misunderstanding what you want ? The problem is, I'm not writing a module that shall be imported, but I'm embedding Python and want to provide a special "built in" module so the final result is a single monolithic binary internally starting the interpreter, creating the module and running user defined scripts. Since boost::python's def function has a return type of void (sure, because there seems to be no type for python functions in bpy), functions can only be exported within a BOOST_PYTHON_MODULE (which I don't have) or when belonging to a class (which led to my workaround). >> global["my_mod"].attr("__dict__")["f1"] = tmp.attr("__dict__")["f1"]; >> > > Why not simply > > global["my_mod"]["f1"] = ... ; ? Does that have the same effect? So I don't have to use the "__dict__" attribute? Didn't know that. As I said, I'm completely new to Python's C API and Boost Python. I'll try that, thanks for the tip. From seefeld at sympatico.ca Tue Jun 23 22:10:01 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue, 23 Jun 2009 16:10:01 -0400 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A413504.3080803@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> Message-ID: <4A413699.10206@sympatico.ca> On 06/23/2009 04:03 PM, Christopher Schramm wrote: > Stefan Seefeld wrote: > >> I read your original mail, but I didn't understand what you are trying >> to achieve. You certainly can export functions to python: >> >> void foo(...); >> >> ... >> >> bpl::def("foo", foo); >> >> >> works just fine. There shouldn't be any need to wrap the function in a >> class (member function), and then create some hidden object to invoke >> it. Or am I misunderstanding what you want ? >> > > The problem is, I'm not writing a module that shall be imported, but I'm > embedding Python and want to provide a special "built in" module so the > final result is a single monolithic binary internally starting the > interpreter, creating the module and running user defined scripts. > I don't see how this affects what you export to Python. > Since boost::python's def function has a return type of void (sure, > because there seems to be no type for python functions in bpy), > functions can only be exported within a BOOST_PYTHON_MODULE (which I > don't have) or when belonging to a class (which led to my workaround). > You need a module into which to inject the symbols you export. That is true no matter the (meta)type of what you export, i.e. classes, functions, etc. Once you have that module set up (via BOOST_PYTHON_MODULE), you can instantiate the newly created Python objects (types) from within C++ code, without having to go through the interpreter, import, eval, or exec. Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From cschramm at shakaweb.org Tue Jun 23 22:43:24 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Tue, 23 Jun 2009 22:43:24 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A413699.10206@sympatico.ca> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> Message-ID: <4A413E6C.9080209@shakaweb.org> Stefan Seefeld wrote: > You need a module into which to inject the symbols you export. That is > true no matter the (meta)type of what you export, i.e. classes, > functions, etc. > Once you have that module set up (via BOOST_PYTHON_MODULE), you can > instantiate the newly created Python objects (types) from within C++ > code, without having to go through the interpreter, import, eval, or exec. So you say I may use BOOST_PYTHON_MODULE to define a module and then immediately work with it? Hm - I failed with that at my very first try. What I did was using BOOST_PYTHON_MODULE and after that calling bpy's import() to add it to my global namespace what failed due to not finding the module. So I assumed I would have to build my module as extension to be able to load it. But as I said, I want to stay monolithic. Could you please show some sample code how you think I can achieve my target using BOOST_PYTHON_MODULE? Here's my goal again: I want to use exec_file() with an already imported module providing access to some C++ functions and classes - all in one binary. From seefeld at sympatico.ca Tue Jun 23 23:00:21 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Tue, 23 Jun 2009 17:00:21 -0400 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A413E6C.9080209@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> Message-ID: <4A414265.7090604@sympatico.ca> On 06/23/2009 04:43 PM, Christopher Schramm wrote: > Stefan Seefeld wrote: > >> You need a module into which to inject the symbols you export. That is >> true no matter the (meta)type of what you export, i.e. classes, >> functions, etc. >> Once you have that module set up (via BOOST_PYTHON_MODULE), you can >> instantiate the newly created Python objects (types) from within C++ >> code, without having to go through the interpreter, import, eval, or exec. >> > > So you say I may use BOOST_PYTHON_MODULE to define a module and then > immediately work with it? Hm - I failed with that at my very first try. > > What I did was using BOOST_PYTHON_MODULE and after that calling bpy's > import() to add it to my global namespace what failed due to not finding > the module. > An alternative is not to use BOOST_PYTHON_MODULE at all, but set up converters in ordinary C++ code. In the following I set up a Python interpreter in my main application, inject a (C++) base class, run a Python script that adds a derived class, then instantiate and run that derived class from C++ code. I this may just be what you want: ------------------------------ namespace bpl = boost::python; // An abstract base class class Base : public boost::noncopyable { public: virtual ~Base() {}; virtual std::string hello() = 0; }; // Familiar Boost.Python wrapper class for Base struct BaseWrap : Base, bpl::wrapper { virtual std::string hello() { return this->get_override("hello")(); } }; int main(int, char **) { Py_Initialize(); // Install C++ <-> Python converters bpl::class_ base("Base"); // Retrieve the main module bpl::object main = bpl::import("__main__"); // Retrieve the main module's namespace bpl::object global(main.attr("__dict__")); // Inject 'Base' type into global dict global["Base"] = base; try { // Run script bpl::exec_file("exec.py", global, global); // Extract 'Derived' class, and instantiate it bpl::object derived = global["Derived"](); // Extract reference-to-base-class Base &d = bpl::extract(derived); // Call 'hello' std::cout << d.hello() << std::endl; } catch (bpl::error_already_set const &) { PyErr_Print(); } } ------------------------------ And the Python side: ------------------------------ class Derived(Base): def hello(self): return 'Hello from Python !' ------------------------------ Easy, isn't it ? Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From gjcarneiro at gmail.com Wed Jun 24 00:51:56 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 23 Jun 2009 23:51:56 +0100 Subject: [C++-sig] Pybindgen/pygccxml integration In-Reply-To: <4A412895.90206@vtiinstruments.com> References: <4A40F01E.3030803@vtiinstruments.com> <4A41064B.7090300@vtiinstruments.com> <4A412895.90206@vtiinstruments.com> Message-ID: 2009/6/23 Ben Fitzpatrick > Gustavo Carneiro wrote: > >> >> >> 2009/6/23 Ben Fitzpatrick > bfitzpatrick at vtiinstruments.com>> >> >> Thanks for the suggestions! >> >> I tried the first one, just as a test. I'd like to do the second >> programatically if I can, but I just wanted to make sure >> annotations were going to fix this. It still seems to be giving me >> the same error: >> (...)/pure_virtual.h:5: WrapperWarning: Parameter 'int * value' >> error (used in int pure_virtual_class::get_value(int * value) >> [member function]): >> > 0xb780326c> >> >> virtual int get_value(int* value) = 0; >> >> Here is what I changed my code to: >> >> // -#- @value(transfer_ownership=true) -#- >> >> virtual int get_value(int* value)=0; >> >> >> Apart from the blank line, seems OK. Do you get a "unused annotation" >> warning? >> >> > That's odd, there wasn't a blank line in my original email. There certainly > isn't one in the file. > I do not get an unused annotation warning, or anything to suggest it > noticed my annotation in the slightest. I am using Python2.4 on Debian Etch, > with Pygccxml 0.9.5. The Python's a bit old, but everything else has been > compiled or installed manually. Well, with Python 2.5 it prints TypeConfigurationError('some message'), which is rather more helpful. You are not seeing the full error message, just the (vague) exception type. And did you run the unit tests (./waf check) ? If the unit tests pass, annotations supposedly are working... > > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From cschramm at shakaweb.org Wed Jun 24 07:52:29 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Wed, 24 Jun 2009 07:52:29 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A414265.7090604@sympatico.ca> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> <4A414265.7090604@sympatico.ca> Message-ID: <4A41BF1D.7080803@shakaweb.org> Stefan Seefeld wrote: > An alternative is not to use BOOST_PYTHON_MODULE at all, but set up > converters in ordinary C++ code. In the following I set up a Python > interpreter in my main application, inject a (C++) base class, run a > Python script that adds a derived class, then instantiate and run that > derived class from C++ code. I this may just be what you want: Sure. That's what I'm doing (just within a submodule below main). But in your example - how do get simple c++ functions into the python scope which aren't members of any class? That's my problem. From merlin66b at gmail.com Wed Jun 24 21:50:22 2009 From: merlin66b at gmail.com (Thomas Berg) Date: Wed, 24 Jun 2009 21:50:22 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A41BF1D.7080803@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> <4A414265.7090604@sympatico.ca> <4A41BF1D.7080803@shakaweb.org> Message-ID: Hi, On Wed, Jun 24, 2009 at 7:52 AM, Christopher Schramm wrote: > > Stefan Seefeld wrote: > > An alternative is not to use BOOST_PYTHON_MODULE at all, but set up > > converters in ordinary C++ code. In the following I set up a Python > > interpreter in my main application, inject a (C++) base class, run a > > Python script that adds a derived class, then instantiate and run that > > derived class from C++ code. I this may just be what you want: > > Sure. That's what I'm doing (just within a submodule below main). But in > your example - how do get simple c++ functions into the python scope > which aren't members of any class? That's my problem. This seems to work: #include #include namespace bp = boost::python; void myfunction() { std::cout << "Hello world!\n"; } int main(int argc, char* argv[]) { Py_Initialize(); bp::object main = bp::import("__main__"); bp::object global = main.attr("__dict__"); bp::object function = bp::object(myfunction); global["function"] = function; bp::exec("function()", global, global); return 0; } Is this what you wanted? Boost python's def returns void, which is natural I think, since you can't assign to the "result" of def in python either. Hope this helps, Thomas From cschramm at shakaweb.org Wed Jun 24 22:16:41 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Wed, 24 Jun 2009 22:16:41 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> <4A414265.7090604@sympatico.ca> <4A41BF1D.7080803@shakaweb.org> Message-ID: <4A4289A9.2070901@shakaweb.org> Thomas Berg wrote: > bp::object function = bp::object(myfunction); Great! And it was that simple... Thanks! From cschramm at shakaweb.org Wed Jun 24 22:26:08 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Wed, 24 Jun 2009 22:26:08 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A4289A9.2070901@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> <4A411FC9.2080004@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> <4A414265.7090604@sympatico.ca> <4A41BF1D.7080803@shakaweb.org> <4A4289A9.2070901@shakaweb.org> Message-ID: <4A428BE0.3070208@shakaweb.org> > Thomas Berg wrote: >> bp::object function = bp::object(myfunction); > > Great! And it was that simple... But wait... Giving that a second thought I don't think that's going to exhaust bpy's full potential. At least I don't see a way to use it's docstring handling or call policies. I'll test it tomorrow. From merlin66b at gmail.com Wed Jun 24 22:59:26 2009 From: merlin66b at gmail.com (Thomas Berg) Date: Wed, 24 Jun 2009 22:59:26 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: <4A428BE0.3070208@shakaweb.org> References: <4A3E1C17.7070502@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> <4A414265.7090604@sympatico.ca> <4A41BF1D.7080803@shakaweb.org> <4A4289A9.2070901@shakaweb.org> <4A428BE0.3070208@shakaweb.org> Message-ID: On Wed, Jun 24, 2009 at 10:26 PM, Christopher Schramm wrote: > > > Thomas Berg wrote: > >> ? ? bp::object function = bp::object(myfunction); > > > > Great! And it was that simple... > > But wait... Giving that a second thought I don't think that's going to > exhaust bpy's full potential. At least I don't see a way to use it's > docstring handling or call policies. > > I'll test it tomorrow. Just discovered a way of doing it properly with "def". By creating a "scope" object you can control where objects created by "def" are inserted, like this: #include #include namespace bp = boost::python; void myfunction() { std::cout << "Hello world!\n"; } int main(int argc, char* argv[]) { Py_Initialize(); bp::object main = bp::import("__main__"); bp::object global = main.attr("__dict__"); { bp::scope sc(main); bp::def("function", myfunction, "function helpstring"); } bp::exec("function()", global, global); bp::exec("help(function)", global, global); return 0; } The scope object sets a global variable which bp::def uses when defining the function. In this case I passed the main module to the scope, so def inserts the definition into it. Getting closer now? Found this out by reading the source code of boost::python::detail::init_module, called by the BOOST_PYTHON_MODULE macro. Cheers, Thomas From mikeowen at llnl.gov Wed Jun 24 23:34:15 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Wed, 24 Jun 2009 14:34:15 -0700 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? Message-ID: Hi again, I've run across another issue confusing me as I try to use pybindgen, in this case it's the treatment of reference parameters in virtual methods where you're allowing the class to be subclassed from python. It looks like the helper class that is generated is making copies of parameters that are being passed as references, which is not what we want. Take the following example C++ classes: class A { public: A(const int val): mVal(val) {} private: int mVal; A(); A(const A& rhs); A& operator=(const A& rhs); }; class B { public: B() {} virtual ~B() {} virtual void some_virtual_method(const A& a) const { std::cerr << &a << std::endl; } private: }; I'd like to be able to create subclasses of B from python, so I try wrapping these classes with the following pybindgen code: mod = Module("ref_param_example") mod.add_include('"classes.hh"') a = mod.add_class("A") b = mod.add_class("B", allow_subclassing=True) a.add_constructor([param("int", "val")]) b.add_constructor([]) b.add_method("some_virtual_method", None, [param("A&", "a")], is_const=True, is_virtual=True) The resulting generated code contains the following: void PyB__PythonHelper::some_virtual_method(A & a) const { PyGILState_STATE __py_gil_state; B *self_obj_before; PyObject *py_retval; PyA *py_A; __py_gil_state = (PyEval_ThreadsInitialized() ? PyGILState_Ensure() : (PyGILState_STATE) 0); if (!PyObject_HasAttrString(m_pyself, (char *) "_some_virtual_method")) { B::some_virtual_method(a); if (PyEval_ThreadsInitialized()) PyGILState_Release(__py_gil_state); return; } self_obj_before = reinterpret_cast< PyB* >(m_pyself)->obj; reinterpret_cast< PyB* >(m_pyself)->obj = const_cast< B* >((const B*) this); py_A = PyObject_New(PyA, &PyA_Type); py_A->obj = new A(a); .... As you can see, the last line quoted here is trying to make a copy of the instance "a" that was passed in. In this example this will not compile because the copy constructor for A is private. In general we don't want to make copies of reference parameters regardless of course. Am I once again missing something here? Is there some way I can declare that the passed parameters for "some_virtual_method" should really be treated as references and not copied? Thanks! Mike. From gjcarneiro at gmail.com Thu Jun 25 00:31:29 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Wed, 24 Jun 2009 23:31:29 +0100 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: 2009/6/24 J. Michael Owen > Hi again, > > I've run across another issue confusing me as I try to use pybindgen, in > this case it's the treatment of reference parameters in virtual methods > where you're allowing the class to be subclassed from python. It looks like > the helper class that is generated is making copies of parameters that are > being passed as references, which is not what we want. Take the following > example C++ classes: > > class A { > public: > A(const int val): mVal(val) {} > private: > int mVal; > A(); > A(const A& rhs); > A& operator=(const A& rhs); > }; > > class B { > public: > B() {} > virtual ~B() {} > virtual void some_virtual_method(const A& a) const { std::cerr << &a << > std::endl; } > private: > }; > > I'd like to be able to create subclasses of B from python, so I try > wrapping these classes with the following pybindgen code: > > mod = Module("ref_param_example") > mod.add_include('"classes.hh"') > a = mod.add_class("A") > b = mod.add_class("B", allow_subclassing=True) > a.add_constructor([param("int", "val")]) > b.add_constructor([]) > b.add_method("some_virtual_method", None, [param("A&", "a")], > is_const=True, is_virtual=True) > > The resulting generated code contains the following: > > void > PyB__PythonHelper::some_virtual_method(A & a) const > { > PyGILState_STATE __py_gil_state; > B *self_obj_before; > PyObject *py_retval; > PyA *py_A; > > __py_gil_state = (PyEval_ThreadsInitialized() ? PyGILState_Ensure() : > (PyGILState_STATE) 0); > if (!PyObject_HasAttrString(m_pyself, (char *) "_some_virtual_method")) > { > B::some_virtual_method(a); > if (PyEval_ThreadsInitialized()) > PyGILState_Release(__py_gil_state); > return; > } > self_obj_before = reinterpret_cast< PyB* >(m_pyself)->obj; > reinterpret_cast< PyB* >(m_pyself)->obj = const_cast< B* >((const B*) > this); > py_A = PyObject_New(PyA, &PyA_Type); > py_A->obj = new A(a); > .... > > As you can see, the last line quoted here is trying to make a copy of the > instance "a" that was passed in. In this example this will not compile > because the copy constructor for A is private. In general we don't want to > make copies of reference parameters regardless of course. > > Am I once again missing something here? Is there some way I can declare > that the passed parameters for "some_virtual_method" should really be > treated as references and not copied? > Sorry, but in this case the answer is no. The virtual method wrapper here receives a parameter from C++ and has to create a Python wrapper for it. The wrapper needs to own the object. The reason is that the Python method that is called may decide to keep a reference to the object. If the Python wrapper kept a "shared" reference to the same C++ object then it could at some point in time be keeping a reference to an object when the caller has already deallocated it. To summarize, the object copy is done because the alternative would be memory unsafe. This is one case where a C++ API blindly designed without considering language bindings may make it difficult to bind later on. Ideally, in this case you should pass a pointer to A, and even better A would have reference counting so that both caller and callee could own a reference to the same object at the same time... -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeowen at llnl.gov Thu Jun 25 01:44:57 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Wed, 24 Jun 2009 16:44:57 -0700 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: I can see what you're worried about here, but as a practical matter it is very useful to be able to pass stuff by reference, and copying it under the hood will break code. For instance, if the virtual functions modify the state of the parameters passed by reference it will just not function correctly when a copy is made. Or in the example I sent when copy constructors are not available it won't even compile! In Boost.Python this can be handled by wrapping the intermediate arguments with the boost::ref() function, which tells Boost.Python to pass the thing along as reference rather than making a copy and on my own head be it if I do something nefarious with that reference. I guess what I would like in this case is the equivalent of an "on my own head be it" option. This is kind of how I think of the pointer passing policy, like with the "caller_owns_return" option. Wouldn't it be reasonable to allow us to specify optionally that references should not be copied, but rather passed along natively? With the default policy still being the more memory safe option of doing the copy? On Jun 24, 2009, at 3:31 PM, Gustavo Carneiro wrote: > Sorry, but in this case the answer is no. The virtual method > wrapper here receives a parameter from C++ and has to create a > Python wrapper for it. The wrapper needs to own the object. The > reason is that the Python method that is called may decide to keep a > reference to the object. If the Python wrapper kept a "shared" > reference to the same C++ object then it could at some point in time > be keeping a reference to an object when the caller has already > deallocated it. > > To summarize, the object copy is done because the alternative would > be memory unsafe. > > This is one case where a C++ API blindly designed without > considering language bindings may make it difficult to bind later > on. Ideally, in this case you should pass a pointer to A, and even > better A would have reference counting so that both caller and > callee could own a reference to the same object at the same time... From gjcarneiro at gmail.com Thu Jun 25 02:03:14 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Thu, 25 Jun 2009 01:03:14 +0100 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: 2009/6/25 J. Michael Owen > I can see what you're worried about here, but as a practical matter it is > very useful to be able to pass stuff by reference, and copying it under the > hood will break code. For instance, if the virtual functions modify the > state of the parameters passed by reference it will just not function > correctly when a copy is made. Or in the example I sent when copy > constructors are not available it won't even compile! > > In Boost.Python this can be handled by wrapping the intermediate arguments > with the boost::ref() function, which tells Boost.Python to pass the thing > along as reference rather than making a copy and on my own head be it if I > do something nefarious with that reference. I guess what I would like in > this case is the equivalent of an "on my own head be it" option. This is > kind of how I think of the pointer passing policy, like with the > "caller_owns_return" option. Wouldn't it be reasonable to allow us to > specify optionally that references should not be copied, but rather passed > along natively? With the default policy still being the more memory safe > option of doing the copy? I can't really fault your logic. I just thought of something that makes some sense. If the wrapper did something like: _wrap_my_virtual_method (A &a) { PyA *py_A; py_A = PyObject_New(PyA, &PyA_Type); py_A->obj = &a; // --- code to call into Python (without consuming a reference to py_A) --- py_A->obj = NULL; Py_DECREF(py_A); } At the end of the virtual method call, if the called python code decided to keep a reference to py_A, it will have a py_A with NULL C++ object and so any method call on it will crash. And this would not need any options. It's always best to do the smart thing when possible without asking too many questions ;-) > > On Jun 24, 2009, at 3:31 PM, Gustavo Carneiro wrote: > > Sorry, but in this case the answer is no. The virtual method wrapper here >> receives a parameter from C++ and has to create a Python wrapper for it. >> The wrapper needs to own the object. The reason is that the Python method >> that is called may decide to keep a reference to the object. If the Python >> wrapper kept a "shared" reference to the same C++ object then it could at >> some point in time be keeping a reference to an object when the caller has >> already deallocated it. >> >> To summarize, the object copy is done because the alternative would be >> memory unsafe. >> >> This is one case where a C++ API blindly designed without considering >> language bindings may make it difficult to bind later on. Ideally, in this >> case you should pass a pointer to A, and even better A would have reference >> counting so that both caller and callee could own a reference to the same >> object at the same time... >> > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeowen at llnl.gov Thu Jun 25 02:11:17 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Wed, 24 Jun 2009 17:11:17 -0700 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: <0862C758-33FE-4F3A-83A8-E96BFA0FAD05@llnl.gov> That sounds great to me! On Jun 24, 2009, at 5:03 PM, Gustavo Carneiro wrote: > > I can't really fault your logic. > > I just thought of something that makes some sense. If the wrapper > did something like: > > _wrap_my_virtual_method (A &a) > { > PyA *py_A; > > py_A = PyObject_New(PyA, &PyA_Type); > py_A->obj = &a; > > // --- code to call into Python (without consuming a reference to > py_A) --- > > py_A->obj = NULL; > Py_DECREF(py_A); > } > > At the end of the virtual method call, if the called python code > decided to keep a reference to py_A, it will have a py_A with NULL C+ > + object and so any method call on it will crash. > > And this would not need any options. It's always best to do the > smart thing when possible without asking too many questions ;-) From nicolaslara at gmail.com Thu Jun 25 05:31:43 2009 From: nicolaslara at gmail.com (Nicolas Lara) Date: Thu, 25 Jun 2009 03:31:43 +0000 Subject: [C++-sig] return_internal_reference & overriding methods in a class. Message-ID: <46ef4bec0906242031i74924c0agc24906f5b4848b08@mail.gmail.com> Hello, I'm having trouble trying to override a method on a class exposed to Python. I cannot modify the original class, so I need to wrap it to change the behaviour of some methods to be Python compatible. The classes look like the following: struct DocumentWrap : Document, wrapper { DocumentWrap() : Document(){ }; DocumentWrap(const Document&) { }; wstring toString() { wstring w(_T("test")); return w; } int test(){ return 1; } }; class_("Hits", init()) .def("doc", &Hits::doc, return_internal_reference<>()); class_("Document") .def("toString", &DocumentWrap::toString) .def("test", &DocumentWrap::test) ; Hits::doc returns a Document. If I do: >>> d = Document() >>> d.test() 1 >>> d.toString() "test" I get the right answer. However, If I obtain the class through Hits I get a reference to a c++ object of type Document (not DocumentWrap) so trying to execute the methods yield: Traceback (most recent call last): File "t.py", line 10, in d.test() Boost.Python.ArgumentError: Python argument types in Document.test(Document) did not match C++ signature: test(DocumentWrap {lvalue}) Traceback (most recent call last): File "t.py", line 10, in d.toString() Boost.Python.ArgumentError: Python argument types in Document.toString(Document) did not match C++ signature: toString(DocumentWrap {lvalue}) Is there any way to get python to understand the internal reference to Document can be treated as a DocumentWrap? Thanks in advance! Best Regards, -- Nicolas Lara Linux user #380134 http://nicolas-lara.blogspot.com/ Public key id: 0x152e7713 at http://subkeys.pgp.net/ From nicolaslara at gmail.com Thu Jun 25 05:46:01 2009 From: nicolaslara at gmail.com (Nicolas Lara) Date: Thu, 25 Jun 2009 03:46:01 +0000 Subject: [C++-sig] return_internal_reference & overriding methods in a class. In-Reply-To: <46ef4bec0906242031i74924c0agc24906f5b4848b08@mail.gmail.com> References: <46ef4bec0906242031i74924c0agc24906f5b4848b08@mail.gmail.com> Message-ID: <46ef4bec0906242046g14d1833co913f52e6ee120081@mail.gmail.com> I tried adding: implicitly_convertible(); implicitly_convertible(); but it doesn't appear to change anything. I would apreciate any idea on thes regard. Thanks again! On Thu, Jun 25, 2009 at 3:31 AM, Nicolas Lara wrote: > Hello, > > I'm having trouble trying to override a method on a class exposed to > Python. I cannot modify the original class, so I need to wrap it to > change the behaviour of some methods to be Python compatible. The > classes look like the following: > > struct DocumentWrap : Document, wrapper > { > ?DocumentWrap() : Document(){ }; > ?DocumentWrap(const Document&) { }; > ?wstring toString() > ?{ > ? ?wstring w(_T("test")); > ? ?return w; > ?} > ?int test(){ > ? ?return 1; > ?} > }; > > ?class_("Hits", init()) > ? ?.def("doc", &Hits::doc, return_internal_reference<>()); > ?class_("Document") > ? ?.def("toString", &DocumentWrap::toString) > ? ?.def("test", &DocumentWrap::test) > ? ?; > > Hits::doc returns a Document. If I do: > >>>> d = Document() >>>> d.test() > 1 >>>> d.toString() > "test" > > I get the right answer. However, If I obtain the class through Hits I > get a reference to a c++ object of type Document (not DocumentWrap) so > trying to execute the methods yield: > > Traceback (most recent call last): > ?File "t.py", line 10, in > ? ?d.test() > Boost.Python.ArgumentError: Python argument types in > ? ?Document.test(Document) > did not match C++ signature: > ? ?test(DocumentWrap {lvalue}) > > Traceback (most recent call last): > ?File "t.py", line 10, in > ? ?d.toString() > Boost.Python.ArgumentError: Python argument types in > ? ?Document.toString(Document) > did not match C++ signature: > ? ?toString(DocumentWrap {lvalue}) > > Is there any way to get python to understand the internal reference to > Document can be treated as a DocumentWrap? > > Thanks in advance! > > Best Regards, > > -- > Nicolas Lara > Linux user #380134 > http://nicolas-lara.blogspot.com/ > Public key id: 0x152e7713 at http://subkeys.pgp.net/ > -- Nicolas Lara Linux user #380134 http://nicolas-lara.blogspot.com/ Public key id: 0x152e7713 at http://subkeys.pgp.net/ From roman.yakovenko at gmail.com Thu Jun 25 06:41:29 2009 From: roman.yakovenko at gmail.com (Roman Yakovenko) Date: Thu, 25 Jun 2009 07:41:29 +0300 Subject: [C++-sig] return_internal_reference & overriding methods in a class. In-Reply-To: <46ef4bec0906242031i74924c0agc24906f5b4848b08@mail.gmail.com> References: <46ef4bec0906242031i74924c0agc24906f5b4848b08@mail.gmail.com> Message-ID: <7465b6170906242141vc6076ccr75e16eb8e297c6d5@mail.gmail.com> On Thu, Jun 25, 2009 at 6:31 AM, Nicolas Lara wrote: > Hello, > > I'm having trouble trying to override a method on a class exposed to > Python. I cannot modify the original class, so I need to wrap it to > change the behaviour of some methods to be Python compatible. The > classes look like the following: > > struct DocumentWrap : Document, wrapper > { > ?DocumentWrap() : Document(){ }; > ?DocumentWrap(const Document&) { }; > ?wstring toString() > ?{ > ? ?wstring w(_T("test")); > ? ?return w; > ?} > ?int test(){ > ? ?return 1; > ?} > }; > > ?class_("Hits", init()) > ? ?.def("doc", &Hits::doc, return_internal_reference<>()); > ?class_("Document") > ? ?.def("toString", &DocumentWrap::toString) > ? ?.def("test", &DocumentWrap::test) > ? ?; > > Hits::doc returns a Document. If I do: > >>>> d = Document() >>>> d.test() > 1 >>>> d.toString() > "test" > > I get the right answer. However, If I obtain the class through Hits I > get a reference to a c++ object of type Document (not DocumentWrap) so > trying to execute the methods yield: > > Traceback (most recent call last): > ?File "t.py", line 10, in > ? ?d.test() > Boost.Python.ArgumentError: Python argument types in > ? ?Document.test(Document) > did not match C++ signature: > ? ?test(DocumentWrap {lvalue}) > > Traceback (most recent call last): > ?File "t.py", line 10, in > ? ?d.toString() > Boost.Python.ArgumentError: Python argument types in > ? ?Document.toString(Document) > did not match C++ signature: > ? ?toString(DocumentWrap {lvalue}) > > Is there any way to get python to understand the internal reference to > Document can be treated as a DocumentWrap? I don't think so. What you ask doesn't make sense. The better solution would be to replace problematic function in another way: struct Doc{ xyz** do_smth(); }; add new global function: boost::python::list do_smth4py( Doc& d){ ///call your function } and register it instead of the original function class_< Doc>(...) .def( "do_smth", &::do_smth4py ) The Python user will not note the difference. HTH -- Roman Yakovenko C++ Python language binding http://www.language-binding.net/ From cschramm at shakaweb.org Thu Jun 25 08:23:08 2009 From: cschramm at shakaweb.org (Christopher Schramm) Date: Thu, 25 Jun 2009 08:23:08 +0200 Subject: [C++-sig] Function handling with Boost In-Reply-To: References: <4A3E1C17.7070502@shakaweb.org> <4A412253.7010308@sympatico.ca> <4A413504.3080803@shakaweb.org> <4A413699.10206@sympatico.ca> <4A413E6C.9080209@shakaweb.org> <4A414265.7090604@sympatico.ca> <4A41BF1D.7080803@shakaweb.org> <4A4289A9.2070901@shakaweb.org> <4A428BE0.3070208@shakaweb.org> Message-ID: <4A4317CC.1030403@shakaweb.org> Thomas Berg wrote: > bp::scope sc(main); > bp::def("function", myfunction, "function helpstring"); Now that looks interesting. I'll implement that and let you know if it leads to any further problems. But it sounds like the perfect solution. > Found this out by reading the source code of > boost::python::detail::init_module, called by the BOOST_PYTHON_MODULE > macro. Good idea. I'm really wondering how someone could develop such a mighty tool like boost python and not even think about writing a useful documentation for it. From sipickles at googlemail.com Thu Jun 25 22:52:44 2009 From: sipickles at googlemail.com (Simon Pickles) Date: Thu, 25 Jun 2009 21:52:44 +0100 Subject: [C++-sig] Uh-oh now I am trying boost python embedding Message-ID: <4A43E39C.1070807@googlemail.com> Hi I've got an embedded python system up and running, for evaluation purposes. I am able to expose c++ modules to python, create c++ classes in python, and extract and use those python types in c++! Cool! However, what if I want to pass a c++ type to a python class? Is there some way to covnert my class, Logger in this case, to an bpy object, or some other way? Here's my code: // main.cpp #include #include #include #include "../logger/logger.h" using namespace boost::python; using namespace std; BOOST_PYTHON_MODULE(hybrid) { class_("Logger") .def("StartLogToFile", &Logger::StartLogToFile) .def("SYS", &Logger::SYS) ; } int main(int, char **) { Py_Initialize(); try { // Get main module and namespace object pyModule_main(handle<>(borrowed(PyImport_AddModule("__main__")))); object pyNamespace_main = pyModule_main.attr("__dict__"); // expose boost python modules inithybrid(); exec("print 'Embedded python....'\n", m_pyNamespace_main); // import another module object pyModule_test = import("test"); object pyNamespace_test = pyModule_test.attr("__dict__"); // c++ object, creating in python, extracted and used in c++! THIS WORKS Logger* log = extract(pyNamespace_test["pyLogger"]); log->SYS("C++ using an object created in python embedded in c++!", 0); Logger* cppLogger = new Logger; cppLogger->StartLogToFile("cppLog.txt"); //Push c++ logger into python namespace THIS DOESN'T! pyNamespace_test["logger"] = cppLogger; object ignored = exec("useCppLogger()", m_pyNamespace_test, m_pyNamespace_test); } catch (error_already_set) { PyErr_Print(); } delete cppLogger; } # test.py import hybrid pyLogger = hybrid.Logger() pyLogger.StartLogToFile("pyLogger.txt") pyLogger.SYS("Python using a python Logger. wow...") def useCppLogger(): # cppLogger has been pushed into module namespace by c++ global cppLogger cppLogger.SYS("Python using a Logger class passed from c++") # Thanks for any help! Simon From seefeld at sympatico.ca Thu Jun 25 22:59:44 2009 From: seefeld at sympatico.ca (Stefan Seefeld) Date: Thu, 25 Jun 2009 16:59:44 -0400 Subject: [C++-sig] Uh-oh now I am trying boost python embedding In-Reply-To: <4A43E39C.1070807@googlemail.com> References: <4A43E39C.1070807@googlemail.com> Message-ID: <4A43E540.1060309@sympatico.ca> On 06/25/2009 04:52 PM, Simon Pickles wrote: > > However, what if I want to pass a c++ type to a python class? Is there > some way to covnert my class, Logger in this case, to an bpy object, > or some other way? Yes, and this conversion happens implicitly, by virtue of the converter that has been created via class_<>. So, the straight-forward way to do this is: Logger *logger = ...; bpl::object l = bpl::ptr(logger); (the ptr() wrapper tells bpl that you want in fact pass by pointer. Without it, bpl makes a copy.) Regards, Stefan -- ...ich hab' noch einen Koffer in Berlin... From camejia at raytheon.com Fri Jun 26 15:48:00 2009 From: camejia at raytheon.com (Christopher A Mejia) Date: Fri, 26 Jun 2009 06:48:00 -0700 Subject: [C++-sig] Simple Boost.Python and NumPy example--help me make it better! Message-ID: <001501c9f664$b87556f0$5f8a1093@dhcp.ess.us.ray.com> Hi, I've been trying to do something that is I would think is a very common task: I'm trying to expose to Python a C++ class with a member function that inputs an array of data. On the Python side, I would like to use NumPy to store this array of data. I've spent many hours searching online for a simple example, but haven't found one. So, I've finally gotten my own simple example working, but I'm not sure I'm handling everything correctly, and I'm sure there's room for improvement...so, I'm posting my example here in the hopes that this group can help me make it better. My goal here is to have short and simple code, at the expense of performance. For example, I'm willing to explicitly copy my data array, if that's what I need to do to make use of existing libraries. The files array_test.hpp and array_test.cpp below contain the class I would like to expose to Python. Note that the member function sum() expects a pointer to a double array and an integer for the lengh of the array. The files array_test2.hpp and array_test2.cpp contain a derived class which inputs a boost::python::numeric::array. The file array_test_ext.cpp uses Boost.Python to expose these classes to Python. Everything appears to be working (the final file array_test.py gives the answer I expect) but I still think there's room for improvement. Here are some questions I have: 1. Is it necessary for me to define a whole new derived class just to add the extra layer on top of the sum() member function? I'm guessing that adding another layer on top of an existing member function must be a pretty common usage pattern so I'm wondering if there's a cleaner way to do it. 2. What is the best way (and the correct syntax) to pass the boost::python::numeric::array into the member function? Right now, I'm just using the syntax for a simple pass-by-value, but maybe I should use pass-by-reference? And should I put a "const" somewhere? 3. I don't think it should be necessary to pass the length of the data array into my function. The boost::python::numeric::array class has an nelements() member function. When I use the nelements() member function, my code compiles, but I get a Python runtime error that NumPy ndarrays don't have an nelements() attribute. What is the correct way to extract the length of the data array from the NumPy array? 4. I must admit that I don't understand . Is there a more efficient way to copy the data from my Python NumPy array into the C++ double * array? Any answers or suggestions for improvement would be most welcome! --Chris // array_test.hpp #ifndef _ARRAY_TEST #define _ARRAY_TEST class array_test { public: double mysqr(double x); double sum(double *x, int n); }; #endif // array_test.cpp #include "array_test.hpp" double array_test::mysqr(double x) { return(x * x); } double array_test::sum(double *x, int n) { double thesum = 0.; for(int i = 0; i < n; i++) { thesum += x[i]; } return(thesum); } // array_test2.hpp #include #include #include "array_test.hpp" class array_test2 : public array_test { public: double sum(boost::python::list o); double sum(boost::python::numeric::array y, int n); }; // array_test2.cpp #include #include "array_test2.hpp" double array_test2::sum(boost::python::list o) { std::size_t n = len(o); double *tmp = new double[n]; for(unsigned int i = 0; i < n; i++) { tmp[i] = boost::python::extract(o[i]); } double thesum = array_test::sum(tmp, n); delete tmp; return(thesum); } double array_test2::sum(boost::python::numeric::array y, int n) { double *tmp = new double[n]; for(int i = 0; i < n; i++) { tmp[i] = boost::python::extract(y[i]); } double thesum = array_test::sum(tmp, n); delete tmp; return(thesum); } // array_test_ext.cpp // This file has been generated by Py++. (and then edited...) #include "boost/python.hpp" #include "/home/cmejia/Python/boost/array_test.hpp" #include "/home/cmejia/Python/boost/array_test2.hpp" namespace bp = boost::python; BOOST_PYTHON_MODULE(array_test_ext){ bp::numeric::array::set_module_and_type("numpy", "ndarray"); bp::class_< array_test >( "array_test" ) .def( "mysqr" , (double ( ::array_test::* )( double ) )( &::array_test::mysqr ) , ( bp::arg("x") ) ) .def( "sum" , (double ( ::array_test::* )( double *,int ) )( &::array_test::sum ) , ( bp::arg("x"), bp::arg("n") ) ); bp::class_< array_test2, bp::bases< array_test > >( "array_test2" ) .def( "sum" , (double ( ::array_test2::* )( bp::list ) )( &::array_test2::sum ) , ( bp::arg("o") ) ) .def( "sum" , (double ( ::array_test2::* )( bp::numeric::array,int ) )( &::array_test2::sum ) , ( bp::arg("y"), bp::arg("n") ) ); } From gjcarneiro at gmail.com Sun Jun 28 17:19:31 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 28 Jun 2009 16:19:31 +0100 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: 2009/6/25 Gustavo Carneiro > > > 2009/6/25 J. Michael Owen > >> I can see what you're worried about here, but as a practical matter it is >> very useful to be able to pass stuff by reference, and copying it under the >> hood will break code. For instance, if the virtual functions modify the >> state of the parameters passed by reference it will just not function >> correctly when a copy is made. Or in the example I sent when copy >> constructors are not available it won't even compile! >> >> In Boost.Python this can be handled by wrapping the intermediate arguments >> with the boost::ref() function, which tells Boost.Python to pass the thing >> along as reference rather than making a copy and on my own head be it if I >> do something nefarious with that reference. I guess what I would like in >> this case is the equivalent of an "on my own head be it" option. This is >> kind of how I think of the pointer passing policy, like with the >> "caller_owns_return" option. Wouldn't it be reasonable to allow us to >> specify optionally that references should not be copied, but rather passed >> along natively? With the default policy still being the more memory safe >> option of doing the copy? > > > I can't really fault your logic. > > I just thought of something that makes some sense. If the wrapper did > something like: > > _wrap_my_virtual_method (A &a) > { > PyA *py_A; > > py_A = PyObject_New(PyA, &PyA_Type); > py_A->obj = &a; > > // --- code to call into Python (without consuming a reference to py_A) > --- > > py_A->obj = NULL; > Py_DECREF(py_A); > } > > At the end of the virtual method call, if the called python code decided to > keep a reference to py_A, it will have a py_A with NULL C++ object and so > any method call on it will crash. > After starting to look at this issue, I realized that this solution is already implemented actually, but I had forgotten! :P You need to tell pybindgen that the reference parameter is INOUT, thus: ReferenceManipulator.add_method('do_manipulate_object', 'void', [Parameter.new('ManipulatedObject&', 'obj', direction=Parameter.DIRECTION_INOUT)], is_virtual=True, is_pure_virtual=True) Then everything works as expected. I am closing the bug by merely adding a unit test to cover this problem. Should work in pybindgen 0.10 as well. Take care, -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From gjcarneiro at gmail.com Sun Jun 28 18:26:24 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Sun, 28 Jun 2009 17:26:24 +0100 Subject: [C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers? In-Reply-To: <99E6647C-40C8-460D-8B39-3351C6F87015@llnl.gov> References: <8595109F5F824F65ADBCBB84FC81A3BB@ncnrwin.ncnr.nist.gov> <99E6647C-40C8-460D-8B39-3351C6F87015@llnl.gov> Message-ID: Thanks Taner and J. Michael Owen. I have a lot to digest from your contributions. The contributions help, but will require some work from me to make them suitable for applying to pybindgen. This issue can be tracked by a bug report from now on, to avoid spamming this list with details: https://bugs.launchpad.net/pybindgen/+bug/384488 2009/6/19 J. Michael Owen > Hello Gustavo and Taner, > Sorry I didn't follow up on this earlier with you guys -- I've been on > travel for a while since I started looking at pybindgen and just returned. > After Gustavo pointed out the python documentation for the sequence method > slots, I thought I would take a look at modifying pybindgen to allow me to > fill in the sequence slots for CppClass much like the numeric operator slots > are handled. I've been able to do this, and am attaching some modified > pybindgen source to this message for your perusal. With these modifications > I can pretty easily expose std::vector so that length, get and set item > (index access), and iteration work. I'll fill in more methods as needed, > but this is a start. > > I took this tack 'cause in my code I have C++ objects that naturally obey > both numeric object protocols (such as +, -, etc) as well as sequence > methods, and I need to reflect those operations into python. I want them to > look like python objects as much as possible, so it's nice to be able to use > [] rather than () for indexing for instance (it's best not to confuse your > users more than necessary). :) > > The way I have this working is if the user adds any of the following > methods to a CppClass ("__len__", "__getitem__", "__setitem__"), then the > corresponding slots in the sequence object protocols are filled in in using > those methods. I think this is fairly natural, since this is how you add > sequence methods to an object you define in python natively. > > With my modifications I can add a container as a cppclass like so: > > mod = Module("example") > mod.add_include("") > mod.add_include('"example.hh"') > std = mod.add_cpp_namespace("std") > vecint = std.add_class("vector", template_parameters=["int"], > custom_template_class_name="vector_of_int") > vecint.add_constructor([]) > vecint.add_constructor([param("int", "size")]) > vecint.add_constructor([param("int", "size"), param("int", "value")]) > vecint.add_method("size", "int", [], custom_name = "__len__") > vecint.add_function_as_method("indexContainer", "int", > [param("std::vector", "self"), > param("int", > "index")], > template_parameters = ["std::vector"], > custom_name = "__getitem__") > vecint.add_function_as_method("assignToPosition", None, > [param("std::vector", "self"), > param("int", > "index"), > param("int", > "value")], > template_parameters = ["std::vector"], > custom_name = "__setitem__") > vecint.add_method("push_back", None, [param("int", "value")], > custom_name="append") > > where "example.hh" contains a few helper methods: > > #include > > namespace std { > > > //------------------------------------------------------------------------------ > // Extract a value from a container. > > //------------------------------------------------------------------------------ > template > inline > typename Container::value_type& > indexContainer(Container& container, > const size_t index) { > try { > return container.at(index); > } catch (out_of_range) { > PyErr_SetString(PyExc_IndexError, "Container index out of range"); > } > } > > > //------------------------------------------------------------------------------ > // Assign to a postion in a container. > > //------------------------------------------------------------------------------ > template > inline > void > assignToPosition(Container& container, > const size_t index, > const typename Container::value_type& value) { > if (index >= container.size()) { > PyErr_SetString(PyExc_IndexError, "Container index out of range"); > } else { > container[index] = value; > } > } > > } > > And the resulting vector can be used from python kind of like you > would naturally expect: > > alastor1{owen}77: python > Python 2.6.1 (r261:67515, Jun 2 2009, 15:40:32) > [GCC 4.3.2 20081007 (Red Hat 4.3.2-7)] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > >>> import example > >>> v = example.std.vector_of_int(20, 5) > >>> len(v) > 20 > >>> [x for x in v] > [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5] > >>> v[10] = 100 > >>> [x for x in v] > [5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 100, 5, 5, 5, 5, 5, 5, 5, 5, 5] > >>> > > In this process I've also made a few other modifications to pybindgen: > > 1. In CppClass I've generalized the binary_numeric_operators to allow the > user to specify python types as one of the arguments. I needed to do this > so I could allow types to define "*" with a python float for instance, > rather than only allowing operators with other CppClass wrapped objects. > > 2. I've also added inplace_numeric_operators as an option for CppClass, > which fills in the appropriate slots in the number methods struct. > > 3. I added name mangling for template arguments, since some of my template > arguments for methods are themselves templates (such as in the example > above). > > The changes I have are made against pybindgen-0.10.0, and you should > compare the files in this tarball to that version. I'm no Python C-API > whiz, so I won't claim these are the most optimal ways to handle these > extensions to pybindgen, but it does work and passes all of pybindgen's > internal tests. I'm continuing to look at how I have to modify pybindgen in > order to meet the needs of wrapping my code, but I don't want to wander too > far afield of the mainline release! If these kinds of mods are not going to > work for want you want to do Gustavo, please let me know! Anyway, check out > these mods as you can and let me know what you think. > > Mike. > > > > On Jun 5, 2009, at 10:00 AM, Gustavo Carneiro wrote: > > > > 2009/6/5 J. Michael Owen > >> Hi Taner, >> Thanks for the suggestion! I actually was trying the same solution, >> except I wasn't sure how to handle the iterators. I'll try it out as you do >> in your example -- are you able to iterate over the vector in python the >> usual way? As in "for x in vec:"? >> >> Two curious things I noted about wrapping "std::vector" myself: >> >> 1. I can't seem to get indexing access to work, even though I wrap an >> indexing operation with the python name "__getitem__" >> >> 2. I also can't get python's len function to work, even if I provide >> "__len__" as follows: >> veci.add_method("size", "int", [], custom_name="__len__") >> >> Have you been able to make either of those methods work? >> > > It won't work because those are special methods in C/C++ side; Python > extension types use a "slots" mechanism for this kind of thing, see http:// > www. python.org/doc/2.6/c-api/typeobj.html > > >> >> Thanks! >> >> Mike. >> >> -- > Gustavo J. A. M. Carneiro > INESC Porto, Telecommunications and Multimedia Unit > "The universe is always one step beyond logic." -- Frank Herbert > > > > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From mikeowen at llnl.gov Mon Jun 29 20:03:44 2009 From: mikeowen at llnl.gov (J. Michael Owen) Date: Mon, 29 Jun 2009 11:03:44 -0700 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: Hi Gustavo, Hmm. I tried this example, but the generated code seems to be still trying to generate a copy. Here's the example source I'm trying expose: #include class A { public: A(const int val): mVal(val) {} private: int mVal; A(); A(const A& rhs); A& operator=(const A& rhs); }; class B { public: B() {} virtual ~B() {} virtual void some_virtual_method(const A& a) const { std::cerr << &a << std::endl; } private: }; I try to expose these objects with the following pybindgen code: mod = Module("ref_param_example") mod.add_include('"classes.hh"') a = mod.add_class("A") b = mod.add_class("B", allow_subclassing=True) a.add_constructor([param("int", "val")]) b.add_constructor([]) b.add_method("some_virtual_method", None, [Parameter.new("A&", "a", direction=Parameter.DIRECTION_INOUT)], is_const=True, is_virtual=True) Now the generated code contains the following function for the helper class required since we're allowing subclassing: void PyB__PythonHelper::some_virtual_method(A & a) const { PyGILState_STATE __py_gil_state; B *self_obj_before; PyObject *py_retval; PyA *py_A; __py_gil_state = (PyEval_ThreadsInitialized() ? PyGILState_Ensure() : (PyGILState_STATE) 0); if (!PyObject_HasAttrString(m_pyself, (char *) "_some_virtual_method")) { B::some_virtual_method(a); if (PyEval_ThreadsInitialized()) PyGILState_Release(__py_gil_state); return; } self_obj_before = reinterpret_cast< PyB* >(m_pyself)->obj; reinterpret_cast< PyB* >(m_pyself)->obj = const_cast< B* >((const B*) this); py_A = PyObject_New(PyA, &PyA_Type); py_A->obj = &(a); py_retval = PyObject_CallMethod(m_pyself, (char *) "_some_virtual_method", (char *) "O", py_A); if (py_retval == NULL) { PyErr_Print(); Py_DECREF(py_A); reinterpret_cast< PyB* >(m_pyself)->obj = self_obj_before; if (PyEval_ThreadsInitialized()) PyGILState_Release(__py_gil_state); return; } if (py_retval != Py_None) { PyErr_SetString(PyExc_TypeError, "function/method should return None"); Py_DECREF(py_retval); Py_DECREF(py_A); reinterpret_cast< PyB* >(m_pyself)->obj = self_obj_before; if (PyEval_ThreadsInitialized()) PyGILState_Release(__py_gil_state); return; } if (py_A->ob_refcnt == 1) py_A->obj = NULL; else{ py_A->obj = new A(a); } Py_DECREF(py_retval); Py_DECREF(py_A); reinterpret_cast< PyB* >(m_pyself)->obj = self_obj_before; if (PyEval_ThreadsInitialized()) PyGILState_Release(__py_gil_state); return; } You can see the line near the end here that is trying to make a copy of the "a" argument with "py_A->obj = new A(a);", which isn't what we want. Why is it trying to do that anyway? This was all generated with pybindgen 0.10.0 by the way -- is there a more recent version that will avoid this problem? Thanks! Mike. On Jun 28, 2009, at 8:19 AM, Gustavo Carneiro wrote: > After starting to look at this issue, I realized that this solution > is already implemented actually, but I had forgotten! :P > > You need to tell pybindgen that the reference parameter is INOUT, > thus: > > ReferenceManipulator.add_method('do_manipulate_object', 'void', > > [Parameter.new('ManipulatedObject&', 'obj', > direction=Parameter.DIRECTION_INOUT)], > is_virtual=True, > is_pure_virtual=True) > > Then everything works as expected. I am closing the bug by merely > adding a unit test to cover this problem. Should work in pybindgen > 0.10 as well. > > Take care, > > -- > Gustavo J. A. M. Carneiro > INESC Porto, Telecommunications and Multimedia Unit > "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: From gcoghlan at mediaware.com.au Tue Jun 30 02:42:49 2009 From: gcoghlan at mediaware.com.au (Garth Coghlan) Date: Tue, 30 Jun 2009 10:42:49 +1000 Subject: [C++-sig] Simple Boost.Python and NumPy example--help me make, it better! In-Reply-To: References: Message-ID: <4A495F89.7020103@mediaware.com.au> Hi Chris, Have you looked at the PyUblas library? This provides a neat interface to C++ with Numpy arrays. On the C++ side you end up with a class that inherits from boost::ublas::matrix, which does no data copying, and which has a nice interface. You can get a pointer to the underlying data as well. You'd still need to wrap your original class, but those wrapper functions would be simpler. Let me know if you need help. Garth cplusplus-sig-request at python.org wrote: > From: "Christopher A Mejia" > > Hi, > > I've been trying to do something that is I would think is a very common > task: I'm trying to expose to Python a C++ class with a member function that > inputs an array of data. On the Python side, I would like to use NumPy to > store this array of data. I've spent many hours searching online for a > simple example, but haven't found one. > From gjcarneiro at gmail.com Tue Jun 30 15:29:39 2009 From: gjcarneiro at gmail.com (Gustavo Carneiro) Date: Tue, 30 Jun 2009 14:29:39 +0100 Subject: [C++-sig] pybindgen: allow_subclassing option causing reference parameters to be copied? In-Reply-To: References: Message-ID: 2009/6/29 J. Michael Owen > Hi Gustavo, > Hmm. I tried this example, but the generated code seems to be still trying > to generate a copy. Here's the example source I'm trying expose: > > #include > > class A { > public: > A(const int val): mVal(val) {} > private: > int mVal; > A(); > A(const A& rhs); > A& operator=(const A& rhs); > }; > > class B { > public: > B() {} > virtual ~B() {} > virtual void some_virtual_method(const A& a) const { std::cerr << &a << > std::endl; } > private: > }; > > I try to expose these objects with the following pybindgen code: > > mod = Module("ref_param_example") > mod.add_include('"classes.hh"') > a = mod.add_class("A") > b = mod.add_class("B", allow_subclassing=True) > a.add_constructor([param("int", "val")]) > b.add_constructor([]) > b.add_method("some_virtual_method", None, [Parameter.new("A&", "a", > direction=Parameter.DIRECTION_INOUT)], is_const=True, is_virtual=True) > > Now the generated code contains the following function for the helper class > required since we're allowing subclassing: > > void > PyB__PythonHelper::some_virtual_method(A & a) const > { > PyGILState_STATE __py_gil_state; > B *self_obj_before; > PyObject *py_retval; > PyA *py_A; > > __py_gil_state = (PyEval_ThreadsInitialized() ? PyGILState_Ensure() : > (PyGILState_STATE) 0); > if (!PyObject_HasAttrString(m_pyself, (char *) "_some_virtual_method")) > { > B::some_virtual_method(a); > if (PyEval_ThreadsInitialized()) > PyGILState_Release(__py_gil_state); > return; > } > self_obj_before = reinterpret_cast< PyB* >(m_pyself)->obj; > reinterpret_cast< PyB* >(m_pyself)->obj = const_cast< B* >((const B*) > this); > py_A = PyObject_New(PyA, &PyA_Type); > py_A->obj = &(a); > py_retval = PyObject_CallMethod(m_pyself, (char *) > "_some_virtual_method", (char *) "O", py_A); > if (py_retval == NULL) { > PyErr_Print(); > Py_DECREF(py_A); > reinterpret_cast< PyB* >(m_pyself)->obj = self_obj_before; > if (PyEval_ThreadsInitialized()) > PyGILState_Release(__py_gil_state); > return; > } > if (py_retval != Py_None) { > PyErr_SetString(PyExc_TypeError, "function/method should return > None"); > Py_DECREF(py_retval); > Py_DECREF(py_A); > reinterpret_cast< PyB* >(m_pyself)->obj = self_obj_before; > if (PyEval_ThreadsInitialized()) > PyGILState_Release(__py_gil_state); > return; > } > if (py_A->ob_refcnt == 1) > py_A->obj = NULL; > else{ > > py_A->obj = new A(a); > } > Py_DECREF(py_retval); > Py_DECREF(py_A); > reinterpret_cast< PyB* >(m_pyself)->obj = self_obj_before; > if (PyEval_ThreadsInitialized()) > PyGILState_Release(__py_gil_state); > return; > } > > You can see the line near the end here that is trying to make a copy of the > "a" argument with "py_A->obj = new A(a);", which isn't what we want. Why is > it trying to do that anyway? This was all generated with pybindgen 0.10.0 > by the way -- is there a more recent version that will avoid this problem? > Ah, right. The thing is, the code tries to make a copy of the object to give to the python wrapper in case it appears the python code kept a reference to the wrapper. I need to fix pybindgen to, if the class has no copy constructor keep the wrapper->obj pointer NULL instead of giving it a copy. The fix has been pushed to the bazaar branch. See https://code.launchpad.net/~gjc/pybindgen/trunk > > Thanks! > > Mike. > > On Jun 28, 2009, at 8:19 AM, Gustavo Carneiro wrote: > > After starting to look at this issue, I realized that this solution is > already implemented actually, but I had forgotten! :P > > You need to tell pybindgen that the reference parameter is INOUT, thus: > > ReferenceManipulator.add_method('do_manipulate_object', 'void', > [Parameter.new('ManipulatedObject&', > 'obj', direction=Parameter.DIRECTION_INOUT)], > is_virtual=True, is_pure_virtual=True) > > Then everything works as expected. I am closing the bug by merely adding a > unit test to cover this problem. Should work in pybindgen 0.10 as well. > > Take care, > > -- > Gustavo J. A. M. Carneiro > INESC Porto, Telecommunications and Multimedia Unit > "The universe is always one step beyond logic." -- Frank Herbert > > > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > http://mail.python.org/mailman/listinfo/cplusplus-sig > -- Gustavo J. A. M. Carneiro INESC Porto, Telecommunications and Multimedia Unit "The universe is always one step beyond logic." -- Frank Herbert -------------- next part -------------- An HTML attachment was scrubbed... URL: