From 515f2 at 163.com Thu Nov 13 14:34:43 2014 From: 515f2 at 163.com (515f2) Date: Thu, 13 Nov 2014 21:34:43 +0800 Subject: [C++-sig] Cplusplus-sig Digest, Vol 70, Issue 5 Message-ID: <8nnx29yvrsigofhbevi24n88.1410066475978@email.android.com> An HTML attachment was scrubbed... URL: -------------- next part -------------- ??????? -------- ???? -------- ????cplusplus-sig-request at python.org ????? 8?31? 18:03 ????cplusplus-sig at python.org ???Cplusplus-sig Digest, Vol 70, Issue 5 >Send Cplusplus-sig mailing list submissions to > cplusplus-sig at python.org > >To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/cplusplus-sig >or, via email, send a message with subject or body 'help' to > cplusplus-sig-request at python.org > >You can reach the person managing the list at > cplusplus-sig-owner at python.org > >When replying, please edit your Subject line so it is more specific >than "Re: Contents of Cplusplus-sig digest..." > > >Today's Topics: > > 1. Boost python extract on Mac OS Mavericks - seg fault > (Brian Bruggeman) > 2. Re: Boost python extract on Mac OS Mavericks - seg fault > (Stefan Seefeld) > > >---------------------------------------------------------------------- > >Message: 1 >Date: Thu, 28 Aug 2014 17:19:07 -0500 >From: Brian Bruggeman >To: cplusplus-sig at python.org >Subject: [C++-sig] Boost python extract on Mac OS Mavericks - seg > fault >Message-ID: <67DFD8DE-AF5F-4F17-AD71-093EEA66C3CD at gmail.com> >Content-Type: text/plain; charset="windows-1252" > >Hi all, > >I am having trouble with a really simple example, and I am hoping someone on this list can help me. > >I have created a github repo for code in question: > > git clone git at github.com:brianbruggeman/boost_python_hello_world.git > >I have also created a stack overflow question: > > http://stackoverflow.com/q/25533329/631199 > >The basic problem is as follows: > > Currently, I am able to compile cleanly, but when executing the code, I receive a segmentation fault. I've narrowed the seg-fault down to the line which actually uses boost::python::extract. > >Code: > > #include > #include > > namespace bp = boost::python; > > // Embedding python > int main(int argc, char** argv) { > int data = 0; > Py_Initialize(); > PyRun_SimpleString("data = 1"); > bp::object module(bp::handle<>(bp::borrowed(PyImport_AddModule("__main__")))); > bp::object dictionary = module.attr("__dict__"); > bp::object data_obj = dictionary["data"]; > // Error: The following line has the segmentation fault... > data = bp::extract(data_obj); > std::cout << "data = " << data << std::endl; > Py_Finalize(); > return 0; > } > >CMakeLists.txt: > > project(hello) > cmake_minimum_required(VERSION 2.8) > > FIND_PACKAGE(PythonInterp) > FIND_PACKAGE(PythonLibs) > FIND_PACKAGE(Boost COMPONENTS python) > > include_directories(${PYTHON_INCLUDE_DIRS} ${Boost_INCLUD_DIRS}) > link_directories(${PYTHON_LIBRARY_DIRS} ${Boost_LIBRARY_DIRS}) > > add_executable(hello say_hello.cpp) > target_link_libraries(hello > ${Boost_LIBRARIES} > ${PYTHON_LIBRARIES}) > >Also for completeness? > >I installed Python and Boost::Python using the following? > > brew install python > brew install boost ?with-python > >Thanks so much in advance! > >Brian >-------------- next part -------------- >An HTML attachment was scrubbed... >URL: > >------------------------------ > >Message: 2 >Date: Sat, 30 Aug 2014 09:38:06 -0400 >From: Stefan Seefeld >To: cplusplus-sig at python.org >Subject: Re: [C++-sig] Boost python extract on Mac OS Mavericks - seg > fault >Message-ID: <5401D3BE.9030700 at seefeld.name> >Content-Type: text/plain; charset=windows-1252 > >On 08/28/2014 06:19 PM, Brian Bruggeman wrote: >> Hi all, >> >> I am having trouble with a really simple example, and I am hoping >> someone on this list can help me. >> >> I have created a github repo for code in question: >> >> git clone git at github.com:brianbruggeman/boost_python_hello_world.git >> >> I have also created a stack overflow question: >> >> http://stackoverflow.com/q/25533329/631199 >> >> The basic problem is as follows: >> >> Currently, I am able to compile cleanly, but when executing the code, >> I receive a segmentation fault. I've narrowed the seg-fault down to >> the line which actually uses boost::python::extract. > >As a first step I suggest wrapping the code in a try-block, to catch >Python exceptions. You may also try extract::check() to verify whether >the object can in fact be converted to the specified type. (In your >version a conversion failure would result in an exception.) >I would also rewrite the code to use more of the Boost.Python API >(import, exec, etc.) > >All that being said, the code compiles and runs fine for me (Fedora 20), >and I don't see anything wrong. > >Regards, >Stefan > > > > > >> >> Code: >> >> #include >> #include >> >> namespace bp = boost::python; >> >> // Embedding python >> int main(int argc, char** argv) { >> int data = 0; >> Py_Initialize(); >> PyRun_SimpleString("data = 1"); >> bp::object >> module(bp::handle<>(bp::borrowed(PyImport_AddModule("__main__")))); >> bp::object dictionary = module.attr("__dict__"); >> bp::object data_obj = dictionary["data"]; >> // Error: The following line has the segmentation fault... >> data = bp::extract(data_obj); >> std::cout << "data = " << data << std::endl; >> Py_Finalize(); >> return 0; >> } >> >> CMakeLists.txt: >> >> project(hello) >> cmake_minimum_required(VERSION 2.8) >> >> FIND_PACKAGE(PythonInterp) >> FIND_PACKAGE(PythonLibs) >> FIND_PACKAGE(Boost COMPONENTS python) >> >> include_directories(${PYTHON_INCLUDE_DIRS} ${Boost_INCLUD_DIRS}) >> link_directories(${PYTHON_LIBRARY_DIRS} ${Boost_LIBRARY_DIRS}) >> >> add_executable(hello say_hello.cpp) >> target_link_libraries(hello >> ${Boost_LIBRARIES} >> ${PYTHON_LIBRARIES}) >> >> Also for completeness? >> >> I installed Python and Boost::Python using the following? >> >> brew install python >> brew install boost ?with-python >> >> Thanks so much in advance! >> >> Brian >> >> >> _______________________________________________ >> Cplusplus-sig mailing list >> Cplusplus-sig at python.org >> https://mail.python.org/mailman/listinfo/cplusplus-sig > > >-- > > ...ich hab' noch einen Koffer in Berlin... > > > >------------------------------ > >Subject: Digest Footer > >_______________________________________________ >Cplusplus-sig mailing list >Cplusplus-sig at python.org >https://mail.python.org/mailman/listinfo/cplusplus-sig > >------------------------------ > >End of Cplusplus-sig Digest, Vol 70, Issue 5 >******************************************** From m at rolle.name Sat Nov 15 22:26:15 2014 From: m at rolle.name (Michael Rolle) Date: Sat, 15 Nov 2014 13:26:15 -0800 Subject: [C++-sig] Build isn't finding the Python functions for shared linke Message-ID: <85B7731278384194AA64CA28DA449FAB@MROLD> Hi, I'm following the instructions for building Boost, as far as I know correctly. When I run 'b2 python link=shared', I get link errors for symbols in the Python interpreter. The problem is that when I build the object files, it is using the 32-bit architecture, whereas my Python is the 64-bit architecture. I haven't found where the magic switch is in the build that tells it to build a 64-bit program. Can you help me with this? Thanks. Michael Rolle -------------- next part -------------- An HTML attachment was scrubbed... URL: From bluescarni at gmail.com Sun Nov 16 19:51:14 2014 From: bluescarni at gmail.com (Francesco Biscani) Date: Sun, 16 Nov 2014 19:51:14 +0100 Subject: [C++-sig] Handling Python exceptions from an exposed C++ function Message-ID: Hello all, I don't know if the subject makes any sense, but hopefully the description of the problem will make things clearer :) I have a template class which I am exposing to Python with different types instantiated. For the sake of simplicity, let's say it is a Vector class. I have various versions of this class exposed, Vector, Vector, etc. and everything works fine. Recently, I thought it would be nice to have, on the Python side, a version of my Vector class that can hold any type of Python object (a bit like the builtin list class). Initially, I thought I could expose the C++ class Vector and would hope that everything would work automagically. Unfortunately, for my own purposes I need that the objects stored in a Vector have C++ semantics wrt copy constructors. This means that I need to be able to do deep copies of the stored object via their copy constructor, for instance, and this is clearly not the case for bp::object. Moreover, I need to be able, e.g., to negate an object with the unary "-" operator, which is not possible for bp::object. Etc. etc. So what I did at this point was to create a C++ class, which I called "bp_object", that just wraps a bp::object and adds the desired features: class bp_object { // ... // Copy/move ctors, assignments, additional operators, etc. // ... private: bp::object m_object; }; Then I added the necessary (trivial) converters to/from Python, registered the conversion with the Boost.Python mechanism, and exposed the Vector class to Python (note that the bp_object class is *not* exposed or visible from Python). Everything works fine until some Python exception is thrown within the bp_object class. For instance, upon insertion into a Vector of a value, I check from C++ that the value being inserted is not zero. I implemented the comparison operator for bp_object as: bool operator==(const bp_object &other) const { return m_object == other.m_object; } This can result in an exception being thrown from Python (e.g., comparison of a numpy array with zero). What happens in this case is that a bp::error_already_set is thrown and not caught by anything, and the execution terminates. I can catch the error_already_set in the comparison operator, print information about the Python exception that was thrown, etc. but what I clearly need to do here is to stop the execution of the program and somehow get back to the Python interpreter and translate the exception. How would I do that? Does what I am trying to do make any sense or is it unnecessarily complicated? It seems like this kind of usage is a mix between extending and embedding, and I am not sure if this is supported at all. Cheers, Francesco. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbosch at astro.princeton.edu Sun Nov 16 20:23:50 2014 From: jbosch at astro.princeton.edu (Jim Bosch) Date: Sun, 16 Nov 2014 14:23:50 -0500 Subject: [C++-sig] Handling Python exceptions from an exposed C++ function In-Reply-To: References: Message-ID: On Sun, Nov 16, 2014 at 1:51 PM, Francesco Biscani wrote: > > This can result in an exception being thrown from Python (e.g., comparison > of a numpy array with zero). What happens in this case is that > a bp::error_already_set is thrown and not caught by anything, and the > execution terminates. > > I can catch the error_already_set in the comparison operator, print > information about the Python exception that was thrown, etc. but what I > clearly need to do here is to stop the execution of the program and somehow > get back to the Python interpreter and translate the exception. > > bp::error_already_set should actually already be doing exactly what you want; the reason it exists is so Boost.Python can catch it and translate it into a Python exception, and any Boost.Python-wrapped function that throws error_already_set should have it caught and translated without having to do anything special. That means your problem is actually one of debugging why that's not working properly. There have been some reports of problems with error_already_set in cases where some C++ compilers/linkers don't recognize an exception as being the same across dynamic library boundaries. I'm not familiar with the details, but that's my best guess as to what's going on. Hopefully that will provide you a starting point for debugging and googling for solutions (or maybe someone else on this list can help diagnose it). Good luck! Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From bluescarni at gmail.com Sun Nov 16 20:47:38 2014 From: bluescarni at gmail.com (Francesco Biscani) Date: Sun, 16 Nov 2014 20:47:38 +0100 Subject: [C++-sig] Handling Python exceptions from an exposed C++ function In-Reply-To: References: Message-ID: Hello Jim, On 16 November 2014 20:23, Jim Bosch wrote: > On Sun, Nov 16, 2014 at 1:51 PM, Francesco Biscani > wrote: > >> >> This can result in an exception being thrown from Python (e.g., >> comparison of a numpy array with zero). What happens in this case is that >> a bp::error_already_set is thrown and not caught by anything, and the >> execution terminates. >> >> I can catch the error_already_set in the comparison operator, print >> information about the Python exception that was thrown, etc. but what I >> clearly need to do here is to stop the execution of the program and somehow >> get back to the Python interpreter and translate the exception. >> >> > bp::error_already_set should actually already be doing exactly what you > want; the reason it exists is so Boost.Python can catch it and translate it > into a Python exception, and any Boost.Python-wrapped function that throws > error_already_set should have it caught and translated without having to do > anything special. > > That means your problem is actually one of debugging why that's not > working properly. There have been some reports of problems with > error_already_set in cases where some C++ compilers/linkers don't recognize > an exception as being the same across dynamic library boundaries. I'm not > familiar with the details, but that's my best guess as to what's going on. > Hopefully that will provide you a starting point for debugging and googling > for solutions (or maybe someone else on this list can help diagnose it). > Thanks for the pointers, I guess I will keep on doing some experiments trying to understand exactly what is going on. I have a Python 3 install available as well, maybe I can try to see if anything changes there. Cheers, Francesco. -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at rolle.name Mon Nov 17 05:10:05 2014 From: m at rolle.name (Michael Rolle) Date: Sun, 16 Nov 2014 20:10:05 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example Message-ID: Hi, I had a hard time building the example/tutorial project. Some of this is, I believe, due to bugs in the distribution, and some is probably my lack of understanding. Some of this I posted to the Boost interest group before I could join this (Python C++) SIG, hoping it would get forwarded on here, so there may be a duplication. 1. Calling bjam in the example/tutorial project failed to even start up the build system. This was because the example/bootstrap.jam pointed to the wrong path for the build system root. Instead of ../../../tools/build/v2, it should be ../../../tools/build/src/kernel. When I changed this, bjam now got past the build system startup. 2. Building the project not only compiles hello.cpp, but it also builds a private copy of the Boost/Python library. So I needed to supply the properties needed to correctly build this library (i.e., link=shared, address-mode=64). Of course, I needed to supply those same properties anyway as part of creating the extension. There's probably a way to change something so that the extension uses the library built in the Boost.Python's own project, or if I have obtained the libraries without having to build them, it would use these. I don't know if you intended to have the tutorial example make its own copies, but it seems a waste of resources to do so. 3. The link for debug mode failed, saying that the .pdb file was in the wrong format (LNK1207). This is a bug, due to an option in the link command '/IMPORTLIB:...hello_ext.pdb'. So the linker is creating the .pdb file as an import library, then complaining that it's not a valid pdb file. I changed '.pdb' to '.lib'. I could also have removed this option entirely, since hello_ext.pyd doesn't have anything to export anyway. 4. Before figuring out that the link was the problem, I changed the /Z7 argument to /Zi in the compile command for hello.cpp. I don't know if this was necessary, or if it was necessary to leave it in place. For now, I just wanted to get it to build. Without /Z7, the debug symbols go into example/tutorial/vc120.pdb. I don't know if the linker found these or not. When I try stepping into the extension, I'll know for sure. Microsoft prefers that .pdb files be used for debug symbols rather than embedding them in the .obj files, so this might be the only real reason to make the change. 4. The link for both release mode failed with two undefined symbols, __imp_DecodePointer and __imp_EncodePointer, which are in kernel32.lib. I tried adding kernel32.lib to the link's inputs. But then it warned that there could be static constructors/destructors not getting called. After much research on this topic, I found that the source of the problem was the /NOENTRY argument in the link command, which has the consequence that the automatic CRT initialization of the DLL doesn't occur. So I remove the /NOENTRY and got not warnings, and I didn't need to add kernerl32.lib either. 5. A minor point. The MACHINE:X64 is redundant. The linker knows it's X64 because of the architecture of the input files and libraries. Nothing wrong with it being there, but it's just clutter in the code. 6. Now bjam was successful in building whatever it wanted to build. It said that hello.test passed. Sounds great, I thought. But I then went into Python and tried 'import hello_ext' and that failed. So I have an issue with the test program passing, while the extension didn't actually work. 7. The problem was that bjam didn't put the hello_ext.pyd file in my Python's lib/site-packages folder. It built the .pyd and .pdb files in the example/tutorial/bin/... staging area, and copied (ONLY) the .pyd file to example/tutorial. So not only did the .pyd file get put in the wrong place, but the .pdb was left out. If I am going to debug my extension (such as with Visual Studio's Python Tools), the .pdb file also needs to be in the lib/site-packages folder. Without the .pdb file, the PTVS debugger will not set breakpoints or step into the extension code. To summarize, the fixes I made in order to build the example and use it in Python were: * Change the build system path in examples/bootstrap.jam. * Change the /IMPORTLIB:....pdb to ....lib in the link command. I could have removed it altogether. * Remove /NOENTRY from the link command. * Manually copy the example/tutorial/bin/.../hello_ext.(pyd,pdb) files to Python's lib/site-packages. This cost me about a whole day's time, and I'm pretty resourceful. And I actually want to build my own extensions in VS instead of bjam, linking to the Boost.Python DLL. I started of by trying this, and my extension crashed. So that's why I went about building the example with bjam, to at least see what a successful deployment looked like. But I'll still have to figure out what went wrong with my original attempt; there may be additional bugs to report. I suppose that a lot of people would have given up building the tutorial example and then given up on Boost.Python altogether. Michael Rolle (408) 313-8149 -------------- next part -------------- An HTML attachment was scrubbed... URL: From raghavendra.jain at gmail.com Mon Nov 17 14:09:02 2014 From: raghavendra.jain at gmail.com (Raghvendra Jain) Date: Mon, 17 Nov 2014 22:09:02 +0900 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: References: Message-ID: Hi Michael, I had written a tutorial on Compiling Boost.Python using Visual Studio. Please check. http://www.sigverse.org/wiki/en/index.php?Compiling%20Boost.Python%20for%20Python%20in%20Visual%20Studio I know I am not directly answering the questions you raised here, but I guess it might help you since this method is working for me and several others. My aim was to embed Python inside the C++ code, so that I can call Python functions. I guess your aim is to call C++ functions from Python. Nevertheless, the process of building the example, is not so different. Please let me know if it helps. I am running on a deadline, else I would have tried to find a concrete solution to your issues. Sorry again. Cheers, Raghav On Mon, Nov 17, 2014 at 1:10 PM, Michael Rolle wrote: > Hi, > > I had a hard time building the example/tutorial project. Some of this is, > I believe, due to bugs in the distribution, and some is probably my lack of > understanding. > > Some of this I posted to the Boost interest group before I could join this > (Python C++) SIG, hoping it would get forwarded on here, so there may be a > duplication. > > 1. Calling bjam in the example/tutorial project failed to even start up > the build system. This was because the example/bootstrap.jam pointed to > the wrong path for the build system root. Instead of > ../../../tools/build/v2, it should be ../../../tools/build/src/kernel. > When I changed this, bjam now got past the build system startup. > > 2. Building the project not only compiles hello.cpp, but it also builds a > private copy of the Boost/Python library. So I needed to supply the > properties needed to correctly build this library (i.e., link=shared, > address-mode=64). Of course, I needed to supply those same properties > anyway as part of creating the extension. > > There's probably a way to change something so that the extension uses the > library built in the Boost.Python's own project, or if I have obtained the > libraries without having to build them, it would use these. I don't know > if you intended to have the tutorial example make its own copies, but it > seems a waste of resources to do so. > > 3. The link for debug mode failed, saying that the .pdb file was in the > wrong format (LNK1207). This is a bug, due to an option in the link > command '/IMPORTLIB:...hello_ext.pdb'. So the linker is creating the .pdb > file as an import library, then complaining that it's not a valid pdb > file. I changed '.pdb' to '.lib'. I could also have removed this option > entirely, since hello_ext.pyd doesn't have anything to export anyway. > > 4. Before figuring out that the link was the problem, I changed the /Z7 > argument to /Zi in the compile command for hello.cpp. I don't know if this > was necessary, or if it was necessary to leave it in place. For now, I > just wanted to get it to build. Without /Z7, the debug symbols go into > example/tutorial/vc120.pdb. I don't know if the linker found these or > not. When I try stepping into the extension, I'll know for sure. > Microsoft prefers that .pdb files be used for debug symbols rather than > embedding them in the .obj files, so this might be the only real reason to > make the change. > > 4. The link for both release mode failed with two undefined symbols, > __imp_DecodePointer and __imp_EncodePointer, which are in kernel32.lib. I > tried adding kernel32.lib to the link's inputs. But then it warned that > there could be static constructors/destructors not getting called. After > much research on this topic, I found that the source of the problem was the > /NOENTRY argument in the link command, which has the consequence that the > automatic CRT initialization of the DLL doesn't occur. So I remove the > /NOENTRY and got not warnings, and I didn't need to add kernerl32.lib > either. > > 5. A minor point. The MACHINE:X64 is redundant. The linker knows it's > X64 because of the architecture of the input files and libraries. Nothing > wrong with it being there, but it's just clutter in the code. > > 6. Now bjam was successful in building whatever it wanted to build. It > said that hello.test passed. Sounds great, I thought. But I then went > into Python and tried 'import hello_ext' and that failed. So I have an > issue with the test program passing, while the extension didn't actually > work. > > 7. The problem was that bjam didn't put the hello_ext.pyd file in my > Python's lib/site-packages folder. It built the .pyd and .pdb files in the > example/tutorial/bin/... staging area, and copied (ONLY) the .pyd file to > example/tutorial. So not only did the .pyd file get put in the wrong > place, but the .pdb was left out. If I am going to debug my extension > (such as with Visual Studio's Python Tools), the .pdb file also needs to be > in the lib/site-packages folder. Without the .pdb file, the PTVS debugger > will not set breakpoints or step into the extension code. > > To summarize, the fixes I made in order to build the example and use it in > Python were: > * Change the build system path in examples/bootstrap.jam. > * Change the /IMPORTLIB:....pdb to ....lib in the link command. I > could have removed it altogether. > * Remove /NOENTRY from the link command. > * Manually copy the example/tutorial/bin/.../hello_ext.(pyd,pdb) files > to Python's lib/site-packages. > > This cost me about a whole day's time, and I'm pretty resourceful. And I > actually want to build my own extensions in VS instead of bjam, linking to > the Boost.Python DLL. I started of by trying this, and my extension > crashed. So that's why I went about building the example with bjam, to at > least see what a successful deployment looked like. But I'll still have to > figure out what went wrong with my original attempt; there may be > additional bugs to report. > > I suppose that a lot of people would have given up building the tutorial > example and then given up on Boost.Python altogether. > > Michael Rolle > (408) 313-8149 > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > https://mail.python.org/mailman/listinfo/cplusplus-sig > -------------- next part -------------- An HTML attachment was scrubbed... URL: From m at rolle.name Mon Nov 17 22:35:11 2014 From: m at rolle.name (Michael Rolle) Date: Mon, 17 Nov 2014 13:35:11 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example References: Message-ID: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> Hi Raghvendra, I'll look at your tutorial shortly. Are you the one responsible for maintaining / bug fixing for Boost.Python? Or would I assume someone else will see my post and act on it? Actually, what I want to know is how I report bugs to whomever will fix them. Thanks. Michael Rolle (408) 313-8149 ----- Original Message ----- From: Raghvendra Jain To: Development of Python/C++ integration Sent: Monday, November 17, 2014 5:09 AM Subject: Re: [C++-sig] Bugs in the Boost.Python tutorial example Hi Michael, I had written a tutorial on Compiling Boost.Python using Visual Studio. Please check. http://www.sigverse.org/wiki/en/index.php?Compiling%20Boost.Python%20for%20Python%20in%20Visual%20Studio I know I am not directly answering the questions you raised here, but I guess it might help you since this method is working for me and several others. My aim was to embed Python inside the C++ code, so that I can call Python functions. I guess your aim is to call C++ functions from Python. Nevertheless, the process of building the example, is not so different. Please let me know if it helps. I am running on a deadline, else I would have tried to find a concrete solution to your issues. Sorry again. Cheers, Raghav On Mon, Nov 17, 2014 at 1:10 PM, Michael Rolle wrote: Hi, I had a hard time building the example/tutorial project. Some of this is, I believe, due to bugs in the distribution, and some is probably my lack of understanding. Some of this I posted to the Boost interest group before I could join this (Python C++) SIG, hoping it would get forwarded on here, so there may be a duplication. 1. Calling bjam in the example/tutorial project failed to even start up the build system. This was because the example/bootstrap.jam pointed to the wrong path for the build system root. Instead of ../../../tools/build/v2, it should be ../../../tools/build/src/kernel. When I changed this, bjam now got past the build system startup. 2. Building the project not only compiles hello.cpp, but it also builds a private copy of the Boost/Python library. So I needed to supply the properties needed to correctly build this library (i.e., link=shared, address-mode=64). Of course, I needed to supply those same properties anyway as part of creating the extension. There's probably a way to change something so that the extension uses the library built in the Boost.Python's own project, or if I have obtained the libraries without having to build them, it would use these. I don't know if you intended to have the tutorial example make its own copies, but it seems a waste of resources to do so. 3. The link for debug mode failed, saying that the .pdb file was in the wrong format (LNK1207). This is a bug, due to an option in the link command '/IMPORTLIB:...hello_ext.pdb'. So the linker is creating the .pdb file as an import library, then complaining that it's not a valid pdb file. I changed '.pdb' to '.lib'. I could also have removed this option entirely, since hello_ext.pyd doesn't have anything to export anyway. 4. Before figuring out that the link was the problem, I changed the /Z7 argument to /Zi in the compile command for hello.cpp. I don't know if this was necessary, or if it was necessary to leave it in place. For now, I just wanted to get it to build. Without /Z7, the debug symbols go into example/tutorial/vc120.pdb. I don't know if the linker found these or not. When I try stepping into the extension, I'll know for sure. Microsoft prefers that .pdb files be used for debug symbols rather than embedding them in the .obj files, so this might be the only real reason to make the change. 4. The link for both release mode failed with two undefined symbols, __imp_DecodePointer and __imp_EncodePointer, which are in kernel32.lib. I tried adding kernel32.lib to the link's inputs. But then it warned that there could be static constructors/destructors not getting called. After much research on this topic, I found that the source of the problem was the /NOENTRY argument in the link command, which has the consequence that the automatic CRT initialization of the DLL doesn't occur. So I remove the /NOENTRY and got not warnings, and I didn't need to add kernerl32.lib either. 5. A minor point. The MACHINE:X64 is redundant. The linker knows it's X64 because of the architecture of the input files and libraries. Nothing wrong with it being there, but it's just clutter in the code. 6. Now bjam was successful in building whatever it wanted to build. It said that hello.test passed. Sounds great, I thought. But I then went into Python and tried 'import hello_ext' and that failed. So I have an issue with the test program passing, while the extension didn't actually work. 7. The problem was that bjam didn't put the hello_ext.pyd file in my Python's lib/site-packages folder. It built the .pyd and .pdb files in the example/tutorial/bin/... staging area, and copied (ONLY) the .pyd file to example/tutorial. So not only did the .pyd file get put in the wrong place, but the .pdb was left out. If I am going to debug my extension (such as with Visual Studio's Python Tools), the .pdb file also needs to be in the lib/site-packages folder. Without the .pdb file, the PTVS debugger will not set breakpoints or step into the extension code. To summarize, the fixes I made in order to build the example and use it in Python were: * Change the build system path in examples/bootstrap.jam. * Change the /IMPORTLIB:....pdb to ....lib in the link command. I could have removed it altogether. * Remove /NOENTRY from the link command. * Manually copy the example/tutorial/bin/.../hello_ext.(pyd,pdb) files to Python's lib/site-packages. This cost me about a whole day's time, and I'm pretty resourceful. And I actually want to build my own extensions in VS instead of bjam, linking to the Boost.Python DLL. I started of by trying this, and my extension crashed. So that's why I went about building the example with bjam, to at least see what a successful deployment looked like. But I'll still have to figure out what went wrong with my original attempt; there may be additional bugs to report. I suppose that a lot of people would have given up building the tutorial example and then given up on Boost.Python altogether. Michael Rolle (408) 313-8149 _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org https://mail.python.org/mailman/listinfo/cplusplus-sig ------------------------------------------------------------------------------ _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org https://mail.python.org/mailman/listinfo/cplusplus-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From jbosch at astro.princeton.edu Mon Nov 17 22:50:34 2014 From: jbosch at astro.princeton.edu (Jim Bosch) Date: Mon, 17 Nov 2014 16:50:34 -0500 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> Message-ID: On Mon, Nov 17, 2014 at 4:35 PM, Michael Rolle wrote: > > Are you the one responsible for maintaining / bug fixing for > Boost.Python? Or would I assume someone else will see my post and act on > it? Actually, what I want to know is how I report bugs to whomever will > fix them. > > To my knowledge, the answer to this question is actually a rather unhappy one - I'm not sure anyone is currently maintaining Boost.Python. I think there may still be general Boost devs who will fix critical bugs, especially if the bug report is accompanied by a patch, but there are a lot of stale tickets in the issue tracker (https://svn.boost.org/trac/boost), and I don't know of anyone who really feels any strong ownership of the library at this stage. I've thought about stepping up to do some of it myself, but given that my day job now involves working with Swig (even though I prefer Boost.Python), it's just not something I can do. I'd love to be corrected by anyone on this list who can say otherwise, of course. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From stefan at seefeld.name Mon Nov 17 23:01:04 2014 From: stefan at seefeld.name (Stefan Seefeld) Date: Mon, 17 Nov 2014 17:01:04 -0500 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> Message-ID: <546A7020.1020802@seefeld.name> On 17/11/14 04:50 PM, Jim Bosch wrote: > On Mon, Nov 17, 2014 at 4:35 PM, Michael Rolle > wrote: > > > > Are you the one responsible for maintaining / bug fixing for > Boost.Python? Or would I assume someone else will see my post and > act on it? Actually, what I want to know is how I report bugs to > whomever will fix them. > > > To my knowledge, the answer to this question is actually a rather > unhappy one - I'm not sure anyone is currently maintaining > Boost.Python. I think there may still be general Boost devs who will > fix critical bugs, especially if the bug report is accompanied by a > patch, but there are a lot of stale tickets in the issue tracker > (https://svn.boost.org/trac/boost), and I don't know of anyone who > really feels any strong ownership of the library at this stage. I've > thought about stepping up to do some of it myself, but given that my > day job now involves working with Swig (even though I prefer > Boost.Python), it's just not something I can do. > > I'd love to be corrected by anyone on this list who can say otherwise, > of course. Unfortunately, I can't. In addition to that, it seems the problems reported in this thread are rather about the build system, which at least some of those with expertise in Boost.Python aren't very familiar with, in particular if it affects platforms they don't use themselves. (Specifically: I'd be happy to help fix issues with the Boost.Python code itself, but I don't feel competent with bjam / Boost.Build, and my knowledge of MSVC is almost non-existent, not to speak of the fact that I have no computer near me running Windows, so I couldn't even test any Windows-specific patches.) Stefan -- ...ich hab' noch einen Koffer in Berlin... From m at rolle.name Mon Nov 17 23:30:45 2014 From: m at rolle.name (Michael Rolle) Date: Mon, 17 Nov 2014 14:30:45 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> <546A7020.1020802@seefeld.name> Message-ID: <6F508CA927B84823AA9B0879F7C45B70@MROLD> Thanks, Stefan. I've created a ticket on the tracker that you provided the link to. It's number 10799. In this case, the fixes should be pretty simple and probably involve the rules for linking DLLs generally and for making Python extensions. One of the fixes just involves a change to a boost-build.jam file, so that it will point to the same build system that the tutorial just told me to create with the bootstrap command. Michael Rolle (408) 313-8149 ----- Original Message ----- From: "Stefan Seefeld" To: Sent: Monday, November 17, 2014 2:01 PM Subject: Re: [C++-sig] Bugs in the Boost.Python tutorial example > On 17/11/14 04:50 PM, Jim Bosch wrote: >> On Mon, Nov 17, 2014 at 4:35 PM, Michael Rolle > > wrote: >> >> >> >> Are you the one responsible for maintaining / bug fixing for >> Boost.Python? Or would I assume someone else will see my post and >> act on it? Actually, what I want to know is how I report bugs to >> whomever will fix them. >> >> >> To my knowledge, the answer to this question is actually a rather >> unhappy one - I'm not sure anyone is currently maintaining >> Boost.Python. I think there may still be general Boost devs who will >> fix critical bugs, especially if the bug report is accompanied by a >> patch, but there are a lot of stale tickets in the issue tracker >> (https://svn.boost.org/trac/boost), and I don't know of anyone who >> really feels any strong ownership of the library at this stage. I've >> thought about stepping up to do some of it myself, but given that my >> day job now involves working with Swig (even though I prefer >> Boost.Python), it's just not something I can do. >> >> I'd love to be corrected by anyone on this list who can say otherwise, >> of course. > > Unfortunately, I can't. In addition to that, it seems the problems > reported in this thread are rather about the build system, which at > least some of those with expertise in Boost.Python aren't very familiar > with, in particular if it affects platforms they don't use themselves. > > (Specifically: I'd be happy to help fix issues with the Boost.Python > code itself, but I don't feel competent with bjam / Boost.Build, and my > knowledge of MSVC is almost non-existent, not to speak of the fact that > I have no computer near me running Windows, so I couldn't even test any > Windows-specific patches.) > > Stefan > > > > -- > > ...ich hab' noch einen Koffer in Berlin... > > _______________________________________________ > Cplusplus-sig mailing list > Cplusplus-sig at python.org > https://mail.python.org/mailman/listinfo/cplusplus-sig From m at rolle.name Mon Nov 17 23:34:03 2014 From: m at rolle.name (Michael Rolle) Date: Mon, 17 Nov 2014 14:34:03 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> Message-ID: <803CFB080E2743B7A35593129512DF59@MROLD> Thanks, Jim. I've created a ticket on the tracker that you provided the link to. It's number 10799. In this case, the fixes should be pretty simple and probably involve the rules for linking DLLs generally and for making Python extensions. One of the fixes just involves a change to a boost-build.jam file, so that it will point to the same build system that the tutorial just told me to create with the bootstrap command. Michael Rolle (408) 313-8149 ----- Original Message ----- From: Jim Bosch To: Development of Python/C++ integration Sent: Monday, November 17, 2014 1:50 PM Subject: Re: [C++-sig] Bugs in the Boost.Python tutorial example On Mon, Nov 17, 2014 at 4:35 PM, Michael Rolle wrote: Are you the one responsible for maintaining / bug fixing for Boost.Python? Or would I assume someone else will see my post and act on it? Actually, what I want to know is how I report bugs to whomever will fix them. To my knowledge, the answer to this question is actually a rather unhappy one - I'm not sure anyone is currently maintaining Boost.Python. I think there may still be general Boost devs who will fix critical bugs, especially if the bug report is accompanied by a patch, but there are a lot of stale tickets in the issue tracker (https://svn.boost.org/trac/boost), and I don't know of anyone who really feels any strong ownership of the library at this stage. I've thought about stepping up to do some of it myself, but given that my day job now involves working with Swig (even though I prefer Boost.Python), it's just not something I can do. I'd love to be corrected by anyone on this list who can say otherwise, of course. Jim ------------------------------------------------------------------------------ _______________________________________________ Cplusplus-sig mailing list Cplusplus-sig at python.org https://mail.python.org/mailman/listinfo/cplusplus-sig -------------- next part -------------- An HTML attachment was scrubbed... URL: From amohr at pixar.com Tue Nov 18 00:54:18 2014 From: amohr at pixar.com (Alex Mohr) Date: Mon, 17 Nov 2014 15:54:18 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: <546A7020.1020802@seefeld.name> References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> <546A7020.1020802@seefeld.name> Message-ID: <546A8AAA.1000403@pixar.com> On 11/17/2014 2:01 PM, Stefan Seefeld wrote: > (Specifically: I'd be happy to help fix issues with the Boost.Python > code itself, but I don't feel competent with bjam / Boost.Build, and my > knowledge of MSVC is almost non-existent, not to speak of the fact that > I have no computer near me running Windows, so I couldn't even test any > Windows-specific patches.) Thanks for offering! Here's a report for a crash bug with a patch that I filed 16 months ago. The formatting is busted in the original report; please see the first comment for a properly formatted test case and patch. https://svn.boost.org/trac/boost/ticket/8978 We've been running this patch successfully since that ticket was filed. Alex From stefan at seefeld.name Tue Nov 18 01:17:51 2014 From: stefan at seefeld.name (Stefan Seefeld) Date: Mon, 17 Nov 2014 19:17:51 -0500 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: <546A8AAA.1000403@pixar.com> References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> <546A7020.1020802@seefeld.name> <546A8AAA.1000403@pixar.com> Message-ID: <546A902F.1060506@seefeld.name> On 17/11/14 06:54 PM, Alex Mohr wrote: > On 11/17/2014 2:01 PM, Stefan Seefeld wrote: >> (Specifically: I'd be happy to help fix issues with the Boost.Python >> code itself, but I don't feel competent with bjam / Boost.Build, and my >> knowledge of MSVC is almost non-existent, not to speak of the fact that >> I have no computer near me running Windows, so I couldn't even test any >> Windows-specific patches.) > > Thanks for offering! Here's a report for a crash bug with a patch > that I filed 16 months ago. The formatting is busted in the original > report; please see the first comment for a properly formatted test > case and patch. > > https://svn.boost.org/trac/boost/ticket/8978 > > We've been running this patch successfully since that ticket was filed. While the issue looks clear (and the patch good), can you attach a test case that would allow me to reproduce the issue (and observe the fix with the patch applied) ? Minimal test cases always accelerate the processing of reported issues. :-) Thanks, Stefan -- ...ich hab' noch einen Koffer in Berlin... From amohr at pixar.com Tue Nov 18 01:48:24 2014 From: amohr at pixar.com (Alex Mohr) Date: Mon, 17 Nov 2014 16:48:24 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: <546A902F.1060506@seefeld.name> References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> <546A7020.1020802@seefeld.name> <546A8AAA.1000403@pixar.com> <546A902F.1060506@seefeld.name> Message-ID: <546A9758.5090701@pixar.com> On 11/17/2014 4:17 PM, Stefan Seefeld wrote: > On 17/11/14 06:54 PM, Alex Mohr wrote: >> On 11/17/2014 2:01 PM, Stefan Seefeld wrote: >>> (Specifically: I'd be happy to help fix issues with the Boost.Python >>> code itself, but I don't feel competent with bjam / Boost.Build, and my >>> knowledge of MSVC is almost non-existent, not to speak of the fact that >>> I have no computer near me running Windows, so I couldn't even test any >>> Windows-specific patches.) >> >> Thanks for offering! Here's a report for a crash bug with a patch >> that I filed 16 months ago. The formatting is busted in the original >> report; please see the first comment for a properly formatted test >> case and patch. >> >> https://svn.boost.org/trac/boost/ticket/8978 >> >> We've been running this patch successfully since that ticket was filed. > > While the issue looks clear (and the patch good), can you attach a test > case that would allow me to reproduce the issue (and observe the fix > with the patch applied) ? > Minimal test cases always accelerate the processing of reported issues. :-) Sorry, the minimal test case is included in the report. I will inline it here also. Alex // C++ #include static void f1(int a0, int a1) { } static void f2(int a0, int a1, int a2) { } BOOST_PYTHON_MODULE(kwargCrash) { boost::python::def("f", f1, (arg("a1")=2)); boost::python::def("f", f2, (arg("a2")=2)); } # Python import kwargCrash kwargCrash.f(0, a1=2) From amohr at pixar.com Tue Nov 18 05:50:06 2014 From: amohr at pixar.com (Alex Mohr) Date: Mon, 17 Nov 2014 20:50:06 -0800 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: <546A902F.1060506@seefeld.name> References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> <546A7020.1020802@seefeld.name> <546A8AAA.1000403@pixar.com> <546A902F.1060506@seefeld.name> Message-ID: <546ACFFE.2060508@pixar.com> On 11/17/14 4:17 PM, Stefan Seefeld wrote: > While the issue looks clear (and the patch good), can you attach a test > case that would allow me to reproduce the issue (and observe the fix > with the patch applied) ? > Minimal test cases always accelerate the processing of reported issues. :-) Also please refer to this message detailing my analysis of the problem with the minimal test case: https://mail.python.org/pipermail/cplusplus-sig/2013-August/017012.html Thanks again! Alex From raghavendra.jain at gmail.com Tue Nov 18 09:28:06 2014 From: raghavendra.jain at gmail.com (Raghvendra Jain) Date: Tue, 18 Nov 2014 17:28:06 +0900 Subject: [C++-sig] Bugs in the Boost.Python tutorial example In-Reply-To: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> References: <4CCCB36314A9484D88AD21AEBB6508D8@MROLD> Message-ID: On Tue, Nov 18, 2014 at 6:35 AM, Michael Rolle wrote: > Are you the one responsible for maintaining / bug fixing for Boost.Python? No man, I am just a normal user. -------------- next part -------------- An HTML attachment was scrubbed... URL: From micdestefano at users.sourceforge.net Fri Nov 21 14:43:14 2014 From: micdestefano at users.sourceforge.net (Michele De Stefano) Date: Fri, 21 Nov 2014 14:43:14 +0100 Subject: [C++-sig] mds-utils 2.1.1 released Message-ID: I am pleased to announce the release of mds-utils 2.1.1 . This release fixes small bugs present into the 2.1.0 release. Users are strongly encouraged to upgrade to this version. Features summary --------------------------- MDS-UTILS provides: 1. a tool for detecting machine endianity. 2. utilities for the Boost uBLAS library. Amongst them, some type traits for detecting different uBLAS matrix types. 3. some useful classes that allow to treat the old C FILE pointer as a C++ stream. 4. C++ wrappers of the main Python objects, independent of those in Boost Python. Wrappers are provided also for NumPy arrays. 5. C++ classes that help on treating Python file objects as C++ streams. 6. a review and refactor of the indexing support in Python extensions. Now access in write mode is supported too. 7. new C++ *to-Python* and *from-Python* converters for some *Boost uBlas* objects and for standard Python objects. These converters do not depend on Boost Python. 8. a new sequence iterator that is able to wrap Python sequences and allows also to modify them. This feature does not depend on Boost.Python. 9. the NDArrayIterator class, that wraps the Numpy C-API iterator and allows easy management of conversions to/from Numpy arrays. 10. some SWIG interface files, for easy integration with SWIG extensions for Python. Each class is a well-documented, small, easy to use and it should never be too difficult to learn to use it. A large percentage of this library makes a heavy usage of the Boost C++ libraries : so, they must be installed on the system. It is assumed that the user is familiar with them. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jsb at dneg.com Tue Nov 25 15:54:58 2014 From: jsb at dneg.com (James Bird) Date: Tue, 25 Nov 2014 14:54:58 +0000 (GMT) Subject: [C++-sig] Exposing std::vector using vector_indexing_suite In-Reply-To: <1998636819.560136.1416927020505.JavaMail.zimbra@dneg.com> Message-ID: <1264670093.563240.1416927298949.JavaMail.zimbra@dneg.com> Hi, We have a simple data structure like this, that we would like to expose via boost python: struct Layer { double value; }; struct Document { std::vector< Layer > layer; }; If we were writing this in pure python, then it could be written as follows: class Layer: def __init__( self ): self.value = 0.0 class Document: def __init__( self ): self.layers = [] We would like our C++ bindings to match the behaviour of python as closely as possible. But we have encountered a scenario in which it doesn't, and we wanted to get some advice on the best way forward. In pure python, the following code produces an output of '123.0': l = Layer( ) d = Document( ) d.layers.append( l ) l.value = 123.0 print d.layers[0].value When exposing C++ classes via boost python (using the vector_indexing_suite for Document.layers), it produces a result of '0.0'. The 'append' is making a copy such that 'l' and 'd.layers[0]' are two separate objects. We've noticed that one way to make this work is by using shared_ptr in our C++ data structure, but we feel we would have to compromise on our C++ API as a result. Do we have any other options? Thanks, James. From jbosch at astro.princeton.edu Wed Nov 26 13:02:58 2014 From: jbosch at astro.princeton.edu (Jim Bosch) Date: Wed, 26 Nov 2014 07:02:58 -0500 Subject: [C++-sig] Exposing std::vector using vector_indexing_suite In-Reply-To: <1264670093.563240.1416927298949.JavaMail.zimbra@dneg.com> References: <1998636819.560136.1416927020505.JavaMail.zimbra@dneg.com> <1264670093.563240.1416927298949.JavaMail.zimbra@dneg.com> Message-ID: On Nov 26, 2014 6:37 AM, "James Bird" wrote: > > Hi, > > We have a simple data structure like this, that we would like to expose via boost python: > > struct Layer > { > double value; > }; > > struct Document > { > std::vector< Layer > layer; > }; > > If we were writing this in pure python, then it could be written as follows: > > class Layer: > def __init__( self ): > self.value = 0.0 > > class Document: > def __init__( self ): > self.layers = [] > > We would like our C++ bindings to match the behaviour of python as closely as possible. But we have encountered a scenario in which it doesn't, and we wanted to get some advice on the best way forward. > > In pure python, the following code produces an output of '123.0': > > l = Layer( ) > d = Document( ) > d.layers.append( l ) > l.value = 123.0 > print d.layers[0].value > > When exposing C++ classes via boost python (using the vector_indexing_suite for Document.layers), it produces a result of '0.0'. > > The 'append' is making a copy such that 'l' and 'd.layers[0]' are two separate objects. We've noticed that one way to make this work is by using shared_ptr in our C++ data structure, but we feel we would have to compromise on our C++ API as a result. Do we have any other options? > I don't think you have any good ones. Any simple C++ version of your Python snippet would also deep-copy the Layer object; append() is copying here simply because it's calling std::vector::push_back(), and that's what the std::vector does. If you want this to work across the C++/Python boundary, you'll need to make it work in pure C++ first, and I do think using shared_ptr seems like the most natural way to do that. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: