From strattonbrazil at gmail.com  Thu Sep  1 15:47:07 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Thu, 1 Sep 2011 06:47:07 -0700
Subject: [C++-sig] sending a c++ class to a python function
In-Reply-To: <4E5D2478.2010706@gmail.com>
References: <CAPVmgLgh-_NgSmHtQviFYqb6hrZcXm2MnJDEHMcrr8jcSZ_=Sg@mail.gmail.com>
	<4E5BD597.3070709@gmail.com>
	<CAPVmgLhG9iiNt=fN-gYg0roRaNB9rKNo447RMBgkbz+xc1t78w@mail.gmail.com>
	<4E5D2478.2010706@gmail.com>
Message-ID: <CAPVmgLhHrnnqD1DrkaLDHpveFm6169eRXh6=vuzf+4EZdr09OA@mail.gmail.com>

That worked.  Thanks.  I had trouble getting it to work as a template,
so I just removed that portion.

//template <typename T*>
inline Scene* get_pointer(QSharedPointer<Scene> const &p) {
   return p.data(); // or whatever
}


On Tue, Aug 30, 2011 at 10:57 AM, Jim Bosch <talljimbo at gmail.com> wrote:
> On 08/30/2011 07:45 AM, Josh Stratton wrote:
>>
>> Oh, okay. ?So I can create a module...
>>
>> #include "scene.h"
>> BOOST_PYTHON_MODULE(scene)
>> {
>> ? ? class_<Scene>("Scene");
>> }
>>
>> and then import it (even though) in my python I normally don't import
>> things I'm not creating. ?I'm assuming this is a boost-python thing to
>> get the class into scope, which gets rid of the "converter found for
>> C++ type: Scene" error.
>
> I don't really understand what you mean; if you want to use code that was
> defined in another Python module, you always have to import it. It's just
> that in this case the module happens to be written in C++.
>
>> from scene import Scene # in my python code
>>
>> In my terminal I get...
>>
>> Error in Python:<class 'Boost.Python.ArgumentError'>: Python argument
>> types in
>> ? ? Mesh.buildByIndex(Scene, PrimitiveParts)
>> did not match C++ signature:
>> ? ? buildByIndex(QSharedPointer<Scene>, PrimitiveParts): ? File
>> "<string>", line 21, in processFile
>>
>> for this function:
>>
>> ? ? static void ? ? ? ? ? ? ? ? ?buildByIndex(SceneP scene,
>> PrimitiveParts parts);
>>
>> The function I'm calling has SceneP typedeffed as a
>> QSharedPointer<Scene> ?and I'm assuming this error is because I haven't
>> made a Scene to QSharedPointer<Scene> ?converter, which should just
>> wrap the Scene object when it comes in requiring a custom conversion
>> function.
>
> This is another case where you probably want to use something other than a
> custom converter (and if you did use a custom converter, you'd want a
> from-python lvalue converter, not a to-python converter, anyways, and those
> are defined differently).
>
> What you probably want to do is tell Boost.Python that QSharedPointer is a
> smart pointer, and tell it to wrap your Scene objects inside one (at least
> if you're going to have to deal with them a lot).
>
> To do that, you'll want to specialize boost::python::pointee and provide a
> get_pointer function:
>
> namespace boost { namespace python {
>
> template <typename T>
> struct pointee< QSharedPointer<T> > {
> ? ?typedef T type;
> };
>
> }}
>
> // in some namespace where ADL will find it...
> template <typename T*>
> inline Scene * get_pointer(QSharedPointer<T> const & p) {
> ? ?return p.get(); // or whatever
> }
>
> Then, when you define the class, use:
>
> class_< Scene, QSharedPointer<Scene> >(...)
>
> You can find more information in the reference documentation:
>
> http://www.boost.org/doc/libs/1_47_0/libs/python/doc/v2/class.html#classes
>
> Good Luck!
>
> Jim
>

From strattonbrazil at gmail.com  Thu Sep  1 16:03:42 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Thu, 1 Sep 2011 07:03:42 -0700
Subject: [C++-sig] copy constructors and multiple instances
Message-ID: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>

In my particular scene I have a Scene class, which operates as kind of
a context for my operations and holds all the meshes in the scene.  I
send my scene down to the python side so import methods will import
into the containers in the scene object.  For example,
addMeshFromFile(scene, fileName) which puts the data from the file
into the scene object.  However, if the scene object isn't be
returned, it seems I'm just copying the scene object (which might be
memory expensive), adding the data to the copy, and throwing the copy
away as I don't return it and it doesn't affect my original scene
object in C++ that I passed in.  I assume this is a fairly common
problem and I need to rework my API somehow like providing a more
robust copy constructor that does a shallow copy of my data
containers.  Still, I might have to reorganize it so I'm not passing
my scene to python.  If an object is copied every time, I think that
might be memory prohibitive.  I'm including my current source from
github.

// header file handling my scene and it's data
https://github.com/strattonbrazil/Sunshine/blob/master/src/scene.h

// functions for creating the bindings to python
https://github.com/strattonbrazil/Sunshine/blob/master/src/python_bindings.cpp

// python file which reads a file and adds it to the scene
https://github.com/strattonbrazil/Sunshine/blob/master/src/objImporter.py

Do I need to change need to change my copy constructor to do a more
shallow copy?  Maybe change my internal scene objects to pointers?
Would this alleviate the memory issue?

From talljimbo at gmail.com  Thu Sep  1 20:11:25 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 01 Sep 2011 11:11:25 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
Message-ID: <4E5FCACD.6020008@gmail.com>

On 09/01/2011 07:03 AM, Josh Stratton wrote:
> In my particular scene I have a Scene class, which operates as kind of
> a context for my operations and holds all the meshes in the scene.  I
> send my scene down to the python side so import methods will import
> into the containers in the scene object.  For example,
> addMeshFromFile(scene, fileName) which puts the data from the file
> into the scene object.  However, if the scene object isn't be
> returned, it seems I'm just copying the scene object (which might be
> memory expensive), adding the data to the copy, and throwing the copy
> away as I don't return it and it doesn't affect my original scene
> object in C++ that I passed in.  I assume this is a fairly common
> problem and I need to rework my API somehow like providing a more
> robust copy constructor that does a shallow copy of my data
> containers.  Still, I might have to reorganize it so I'm not passing
> my scene to python.  If an object is copied every time, I think that
> might be memory prohibitive.  I'm including my current source from
> github.
>
> // header file handling my scene and it's data
> https://github.com/strattonbrazil/Sunshine/blob/master/src/scene.h
>
> // functions for creating the bindings to python
> https://github.com/strattonbrazil/Sunshine/blob/master/src/python_bindings.cpp
>
> // python file which reads a file and adds it to the scene
> https://github.com/strattonbrazil/Sunshine/blob/master/src/objImporter.py
>
> Do I need to change need to change my copy constructor to do a more
> shallow copy?  Maybe change my internal scene objects to pointers?
> Would this alleviate the memory issue?

A couple of comments:

- I'm a bit confused by your python_bindings.cpp file; did you paste 
together several files?  If not, I've never seen anyone try to define 
more than one module in a single shared library, and I don't it's 
supposed to work.  You can wrap all of your objects in a single Python 
module - the body of a BOOST_PYTHON_MODULE block is just a regular 
function block, so you can include as many calls to class_ (and other 
things) in there as you like.  This might be part of the confusion about 
needing to import modules from one of your earlier emails.

- You shouldn't have to do anything to the copy constructor to make this 
work.  Boost.Python is perfectly happy to pass things by reference or by 
pointer.  It could be that your QSharedPointers are getting in the way 
of this.  What is the signature of Mesh::buildByIndex?  If the Scene is 
passed by QSharedPointer, that might force a copy, because Boost.Python 
only knows how to dereference the pointer, not copy it (in other words, 
it doesn't want to assume it can copy the smart pointer, because some 
smart pointers like auto_ptr can't be copied, so it's making a new 
QSharedPointer from a copy-constructed object).  If you can change it, 
just make it take a Scene by reference - that will almost certainly 
avoid making copies.  Alternately, if you can use boost::shared_ptr 
instead, that will also solve all of your problems.  Unfortunately 
support for non-boost shared pointers isn't great in Boost.Python right now.

- If you really want to ensure Boost.Python doesn't make unnecessary 
copies of Scene, wrap it with:

class_<Scene,QSharedPointer<Scene>,boost::noncopyable>(...)

That will cause things that require copies to produce compiler errors, 
and you can follow those errors to see exactly where the copy is 
required.  It will also keep you from returning Scene objects by value, 
however, but it sounds like that might be desirable.

Jim



From strattonbrazil at gmail.com  Thu Sep  1 22:01:52 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Thu, 1 Sep 2011 13:01:52 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <4E5FCACD.6020008@gmail.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
Message-ID: <CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>

>> In my particular scene I have a Scene class, which operates as kind of
>> a context for my operations and holds all the meshes in the scene. ?I
>> send my scene down to the python side so import methods will import
>> into the containers in the scene object. ?For example,
>> addMeshFromFile(scene, fileName) which puts the data from the file
>> into the scene object. ?However, if the scene object isn't be
>> returned, it seems I'm just copying the scene object (which might be
>> memory expensive), adding the data to the copy, and throwing the copy
>> away as I don't return it and it doesn't affect my original scene
>> object in C++ that I passed in. ?I assume this is a fairly common
>> problem and I need to rework my API somehow like providing a more
>> robust copy constructor that does a shallow copy of my data
>> containers. ?Still, I might have to reorganize it so I'm not passing
>> my scene to python. ?If an object is copied every time, I think that
>> might be memory prohibitive. ?I'm including my current source from
>> github.
>>
>> // header file handling my scene and it's data
>> https://github.com/strattonbrazil/Sunshine/blob/master/src/scene.h
>>
>> // functions for creating the bindings to python
>>
>> https://github.com/strattonbrazil/Sunshine/blob/master/src/python_bindings.cpp
>>
>> // python file which reads a file and adds it to the scene
>> https://github.com/strattonbrazil/Sunshine/blob/master/src/objImporter.py
>>
>> Do I need to change need to change my copy constructor to do a more
>> shallow copy? ?Maybe change my internal scene objects to pointers?
>> Would this alleviate the memory issue?
>
> A couple of comments:
>
> - I'm a bit confused by your python_bindings.cpp file; did you paste
> together several files? ?If not, I've never seen anyone try to define more
> than one module in a single shared library, and I don't it's supposed to
> work. ?You can wrap all of your objects in a single Python module - the body
> of a BOOST_PYTHON_MODULE block is just a regular function block, so you can
> include as many calls to class_ (and other things) in there as you like.
> ?This might be part of the confusion about needing to import modules from
> one of your earlier emails.

I'm currently just embedding python, but I eventually wanted to extend
it down the road.  I guess putting everything under the same macro
makes more sense.  One thing I could find briefly skimming over the
docs is creating submodules.  If the BOOST_PYTHON_MODULE is named
"foo", is there some macro/function I can put inside of that handles
submodules, so classes imported under it are imported in python such
as "from foo.bar import SomeClass"?

> - You shouldn't have to do anything to the copy constructor to make this
> work. ?Boost.Python is perfectly happy to pass things by reference or by
> pointer. ?It could be that your QSharedPointers are getting in the way of
> this. ?What is the signature of Mesh::buildByIndex? ?If the Scene is passed
> by QSharedPointer, that might force a copy, because Boost.Python only knows
> how to dereference the pointer, not copy it (in other words, it doesn't want
> to assume it can copy the smart pointer, because some smart pointers like
> auto_ptr can't be copied, so it's making a new QSharedPointer from a
> copy-constructed object). ?If you can change it, just make it take a Scene
> by reference - that will almost certainly avoid making copies. ?Alternately,
> if you can use boost::shared_ptr instead, that will also solve all of your
> problems. ?Unfortunately support for non-boost shared pointers isn't great
> in Boost.Python right now.

Switching to boost's shared_ptr shouldn't be difficult.
Mesh::buildByIndex does take a shared-pointer, but if I understand
correctly, switching to boost::shared_ptr<Scene> should solve my
issue.

> - If you really want to ensure Boost.Python doesn't make unnecessary copies
> of Scene, wrap it with:
>
> class_<Scene,QSharedPointer<Scene>,boost::noncopyable>(...)
>
> That will cause things that require copies to produce compiler errors, and
> you can follow those errors to see exactly where the copy is required. ?It
> will also keep you from returning Scene objects by value, however, but it
> sounds like that might be desirable.

I saw this in the docs and seems to be what I need.  Like you said, at
least I'd get compiling error instead of guessing what is going on in
the background.  Thanks.

From talljimbo at gmail.com  Thu Sep  1 22:52:56 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 01 Sep 2011 13:52:56 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
Message-ID: <4E5FF0A8.1080004@gmail.com>

On 09/01/2011 01:01 PM, Josh Stratton wrote:
>> - I'm a bit confused by your python_bindings.cpp file; did you paste
>> together several files?  If not, I've never seen anyone try to define more
>> than one module in a single shared library, and I don't it's supposed to
>> work.  You can wrap all of your objects in a single Python module - the body
>> of a BOOST_PYTHON_MODULE block is just a regular function block, so you can
>> include as many calls to class_ (and other things) in there as you like.
>>   This might be part of the confusion about needing to import modules from
>> one of your earlier emails.
>
> I'm currently just embedding python, but I eventually wanted to extend
> it down the road.  I guess putting everything under the same macro
> makes more sense.  One thing I could find briefly skimming over the
> docs is creating submodules.  If the BOOST_PYTHON_MODULE is named
> "foo", is there some macro/function I can put inside of that handles
> submodules, so classes imported under it are imported in python such
> as "from foo.bar import SomeClass"?
>

To do that, you just need to add some pure Python and a directory 
structure.  If "foo" is a directory, and bar.so is a Boost.Python 
extension module (suffix is different on Windows, of course), you just 
need to have:

foo/__init__.py
foo/bar.so

But just like you can't have a pure-Python package+module structure in a 
single file, you can't do it in C++ either; you need to have a directory 
and the __init__.py file.

>> - You shouldn't have to do anything to the copy constructor to make this
>> work.  Boost.Python is perfectly happy to pass things by reference or by
>> pointer.  It could be that your QSharedPointers are getting in the way of
>> this.  What is the signature of Mesh::buildByIndex?  If the Scene is passed
>> by QSharedPointer, that might force a copy, because Boost.Python only knows
>> how to dereference the pointer, not copy it (in other words, it doesn't want
>> to assume it can copy the smart pointer, because some smart pointers like
>> auto_ptr can't be copied, so it's making a new QSharedPointer from a
>> copy-constructed object).  If you can change it, just make it take a Scene
>> by reference - that will almost certainly avoid making copies.  Alternately,
>> if you can use boost::shared_ptr instead, that will also solve all of your
>> problems.  Unfortunately support for non-boost shared pointers isn't great
>> in Boost.Python right now.
>
> Switching to boost's shared_ptr shouldn't be difficult.
> Mesh::buildByIndex does take a shared-pointer, but if I understand
> correctly, switching to boost::shared_ptr<Scene>  should solve my
> issue.

Yup.  With boost::shared_ptr, you don't even need to include it as a 
template argument of class_, if you instead add the line:

boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();

This allows you to return shared_ptr to Python, but because a 
boost::shared_ptr can use an arbitrary deleter, Boost.Python can convert 
any wrapped Scene object (even one that doesn't hold a 
shared_ptr<Scene>) into a shared_ptr<Scene> with correct reference counting.

If you do include boost::shared_ptr<Scene> in the class_ template 
arguments, all Scene objects constructed in Python will be inside a 
shared_ptr, but because of the above, that isn't necessary unless you 
have functions that need to reset the shared_ptr itself.

Jim

From strattonbrazil at gmail.com  Fri Sep  2 04:31:24 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Thu, 1 Sep 2011 19:31:24 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <4E5FF0A8.1080004@gmail.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com>
Message-ID: <CAPVmgLi_BFUhoroyZ=YY4COsvxn8zo-eqxkYHBn6jgHDwsfVng@mail.gmail.com>

BOOST_PYTHON_MODULE(scene)
{
    class_<Scene, boost::noncopyable>("Scene");
}

// later calling this
boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
PyImport_AppendInittab("scene", &initscene);

So is this all I really need for the mapping then?  I'm getting an
error trying to send the Scene object down to python.

Error in Python: <type 'exceptions.TypeError'>: No to_python
(by-value) converter found for C++ type: Scene

On Thu, Sep 1, 2011 at 1:52 PM, Jim Bosch <talljimbo at gmail.com> wrote:
> On 09/01/2011 01:01 PM, Josh Stratton wrote:
>>>
>>> - I'm a bit confused by your python_bindings.cpp file; did you paste
>>> together several files? ?If not, I've never seen anyone try to define
>>> more
>>> than one module in a single shared library, and I don't it's supposed to
>>> work. ?You can wrap all of your objects in a single Python module - the
>>> body
>>> of a BOOST_PYTHON_MODULE block is just a regular function block, so you
>>> can
>>> include as many calls to class_ (and other things) in there as you like.
>>> ?This might be part of the confusion about needing to import modules from
>>> one of your earlier emails.
>>
>> I'm currently just embedding python, but I eventually wanted to extend
>> it down the road. ?I guess putting everything under the same macro
>> makes more sense. ?One thing I could find briefly skimming over the
>> docs is creating submodules. ?If the BOOST_PYTHON_MODULE is named
>> "foo", is there some macro/function I can put inside of that handles
>> submodules, so classes imported under it are imported in python such
>> as "from foo.bar import SomeClass"?
>>
>
> To do that, you just need to add some pure Python and a directory structure.
> ?If "foo" is a directory, and bar.so is a Boost.Python extension module
> (suffix is different on Windows, of course), you just need to have:
>
> foo/__init__.py
> foo/bar.so
>
> But just like you can't have a pure-Python package+module structure in a
> single file, you can't do it in C++ either; you need to have a directory and
> the __init__.py file.
>
>>> - You shouldn't have to do anything to the copy constructor to make this
>>> work. ?Boost.Python is perfectly happy to pass things by reference or by
>>> pointer. ?It could be that your QSharedPointers are getting in the way of
>>> this. ?What is the signature of Mesh::buildByIndex? ?If the Scene is
>>> passed
>>> by QSharedPointer, that might force a copy, because Boost.Python only
>>> knows
>>> how to dereference the pointer, not copy it (in other words, it doesn't
>>> want
>>> to assume it can copy the smart pointer, because some smart pointers like
>>> auto_ptr can't be copied, so it's making a new QSharedPointer from a
>>> copy-constructed object). ?If you can change it, just make it take a
>>> Scene
>>> by reference - that will almost certainly avoid making copies.
>>> ?Alternately,
>>> if you can use boost::shared_ptr instead, that will also solve all of
>>> your
>>> problems. ?Unfortunately support for non-boost shared pointers isn't
>>> great
>>> in Boost.Python right now.
>>
>> Switching to boost's shared_ptr shouldn't be difficult.
>> Mesh::buildByIndex does take a shared-pointer, but if I understand
>> correctly, switching to boost::shared_ptr<Scene> ?should solve my
>> issue.
>
> Yup. ?With boost::shared_ptr, you don't even need to include it as a
> template argument of class_, if you instead add the line:
>
> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
>
> This allows you to return shared_ptr to Python, but because a
> boost::shared_ptr can use an arbitrary deleter, Boost.Python can convert any
> wrapped Scene object (even one that doesn't hold a shared_ptr<Scene>) into a
> shared_ptr<Scene> with correct reference counting.
>
> If you do include boost::shared_ptr<Scene> in the class_ template arguments,
> all Scene objects constructed in Python will be inside a shared_ptr, but
> because of the above, that isn't necessary unless you have functions that
> need to reset the shared_ptr itself.
>
> Jim
>

From hans_meine at gmx.net  Fri Sep  2 10:21:06 2011
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 2 Sep 2011 10:21:06 +0200
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <4E5FF0A8.1080004@gmail.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com>
Message-ID: <201109021021.06815.hans_meine@gmx.net>

Am Donnerstag, 1. September 2011, 22:52:56 schrieb Jim Bosch:
> Yup.  With boost::shared_ptr, you don't even need to include it as a
> template argument of class_, if you instead add the line:
> 
> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
> 
> This allows you to return shared_ptr to Python, but because a
> boost::shared_ptr can use an arbitrary deleter, Boost.Python can convert
> any wrapped Scene object (even one that doesn't hold a
> shared_ptr<Scene>) into a shared_ptr<Scene> with correct reference
> counting.
> 
> If you do include boost::shared_ptr<Scene> in the class_ template
> arguments, all Scene objects constructed in Python will be inside a
> shared_ptr, but because of the above, that isn't necessary unless you
> have functions that need to reset the shared_ptr itself.

Jim, this is the best write-up on the shared_ptr support I read _in all the 
years_.  The above matches my current mental model of the topic, but it took 
me a long time to understand this.  The custom deleter part (rarely mentioned 
and probably not well known even among shared_ptr users) is absolutely crucial 
information to people with a solid C++ (and Python) background trying to 
understand BPL behavior in this respect.  Looking forward to you improving the 
docs. :-)

Best,
  Hans

From hans_meine at gmx.net  Fri Sep  2 10:24:17 2011
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 2 Sep 2011 10:24:17 +0200
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <CAPVmgLi_BFUhoroyZ=YY4COsvxn8zo-eqxkYHBn6jgHDwsfVng@mail.gmail.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com>
	<CAPVmgLi_BFUhoroyZ=YY4COsvxn8zo-eqxkYHBn6jgHDwsfVng@mail.gmail.com>
Message-ID: <201109021024.17490.hans_meine@gmx.net>

Am Freitag, 2. September 2011, 04:31:24 schrieb Josh Stratton:
> BOOST_PYTHON_MODULE(scene)
> {
>     class_<Scene, boost::noncopyable>("Scene");
> }
> 
> // later calling this
> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
> PyImport_AppendInittab("scene", &initscene);
> 
> So is this all I really need for the mapping then?

register_ptr_to_python deals with the boost::shared_ptr around a Scene, but of 
course you still need a converter for the latter, e.g. as registered by using 
class_ to describe your class.

HTH
  Hans

From strattonbrazil at gmail.com  Fri Sep  2 11:44:22 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Fri, 2 Sep 2011 02:44:22 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <201109021024.17490.hans_meine@gmx.net>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com>
	<CAPVmgLi_BFUhoroyZ=YY4COsvxn8zo-eqxkYHBn6jgHDwsfVng@mail.gmail.com>
	<201109021024.17490.hans_meine@gmx.net>
Message-ID: <CAPVmgLgJweqWbcuktFpK9=RYJ6fxMNF5fUVKeNdN3vy5oa_DZA@mail.gmail.com>

This is where I'm calling the function inside Scene, by the way, and
getting that error.

object processFileFunc = _pyMainModule.attr("MeshImporter").attr("processFile");
processFileFunc(this, fileName); // right here, where the python func
takes a scene object and filename

I haven't touched this part of the code when since I was using
QSharedPointers, so I assume there's something I'm still not doing to
setup this mapping.

On Fri, Sep 2, 2011 at 1:24 AM, Hans Meine <hans_meine at gmx.net> wrote:
> Am Freitag, 2. September 2011, 04:31:24 schrieb Josh Stratton:
>> BOOST_PYTHON_MODULE(scene)
>> {
>> ? ? class_<Scene, boost::noncopyable>("Scene");
>> }
>>
>> // later calling this
>> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
>> PyImport_AppendInittab("scene", &initscene);
>>
>> So is this all I really need for the mapping then?
>
> register_ptr_to_python deals with the boost::shared_ptr around a Scene, but of
> course you still need a converter for the latter, e.g. as registered by using
> class_ to describe your class.
>
> HTH
> ?Hans
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From dave at boostpro.com  Fri Sep  2 21:19:58 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Fri, 02 Sep 2011 11:19:58 -0800
Subject: [C++-sig] copy constructors and multiple instances
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com>
Message-ID: <m21uvyg3a9.fsf@boostpro.com>


on Thu Sep 01 2011, Jim Bosch <talljimbo-AT-gmail.com> wrote:

> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
>
> This allows you to return shared_ptr to Python, but because a
> boost::shared_ptr can use an arbitrary deleter, Boost.Python can
> convert any wrapped Scene object (even one that doesn't hold a
> shared_ptr<Scene>) into a shared_ptr<Scene> with correct reference
> counting.

I might have forgotten by now, but I don't think you need to register
shared_ptr, and I don't think the above will have any effect.  IIRC I
made sure shared_ptr support "just works" out-of-the-box.

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From strattonbrazil at gmail.com  Fri Sep  2 21:47:01 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Fri, 2 Sep 2011 12:47:01 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <m21uvyg3a9.fsf@boostpro.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com> <m21uvyg3a9.fsf@boostpro.com>
Message-ID: <CAPVmgLiGPf9S5uD3OEk0o3GtCyy5wmrGd-h-7_31iDCAVYVRRA@mail.gmail.com>

Well, right now I'm just passing "this" from inside the scene object,
so does that need to be wrapped in a shared pointer?

On Fri, Sep 2, 2011 at 12:19 PM, Dave Abrahams <dave at boostpro.com> wrote:
>
> on Thu Sep 01 2011, Jim Bosch <talljimbo-AT-gmail.com> wrote:
>
>> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
>>
>> This allows you to return shared_ptr to Python, but because a
>> boost::shared_ptr can use an arbitrary deleter, Boost.Python can
>> convert any wrapped Scene object (even one that doesn't hold a
>> shared_ptr<Scene>) into a shared_ptr<Scene> with correct reference
>> counting.
>
> I might have forgotten by now, but I don't think you need to register
> shared_ptr, and I don't think the above will have any effect. ?IIRC I
> made sure shared_ptr support "just works" out-of-the-box.
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From dave at boostpro.com  Fri Sep  2 23:15:32 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Fri, 02 Sep 2011 13:15:32 -0800
Subject: [C++-sig] copy constructors and multiple instances
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com> <m21uvyg3a9.fsf@boostpro.com>
	<CAPVmgLiGPf9S5uD3OEk0o3GtCyy5wmrGd-h-7_31iDCAVYVRRA@mail.gmail.com>
Message-ID: <m2bov2ejd7.fsf@boostpro.com>


on Fri Sep 02 2011, Josh Stratton <strattonbrazil-AT-gmail.com> wrote:

> Well, right now I'm just passing "this" from inside the scene object,
> so does that need to be wrapped in a shared pointer?

That's a pretty vague description of what you're doing, so it's hard to
say.  I suggest you reduce your question to a minimal case that doesn't
involve any of your domain-specific stuff.  Then it will be easy to
answer.

>
> On Fri, Sep 2, 2011 at 12:19 PM, Dave Abrahams <dave at boostpro.com> wrote:
>>
>> on Thu Sep 01 2011, Jim Bosch <talljimbo-AT-gmail.com> wrote:
>>
>>> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
>>>
>>> This allows you to return shared_ptr to Python, but because a
>>> boost::shared_ptr can use an arbitrary deleter, Boost.Python can
>>> convert any wrapped Scene object (even one that doesn't hold a
>>> shared_ptr<Scene>) into a shared_ptr<Scene> with correct reference
>>> counting.
>>
>> I might have forgotten by now, but I don't think you need to register
>> shared_ptr, and I don't think the above will have any effect. ?IIRC I
>> made sure shared_ptr support "just works" out-of-the-box.
>>
>> --
>> Dave Abrahams
>> BoostPro Computing
>> http://www.boostpro.com>
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig>

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From strattonbrazil at gmail.com  Sat Sep  3 05:27:18 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Fri, 2 Sep 2011 20:27:18 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <m2bov2ejd7.fsf@boostpro.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com> <m21uvyg3a9.fsf@boostpro.com>
	<CAPVmgLiGPf9S5uD3OEk0o3GtCyy5wmrGd-h-7_31iDCAVYVRRA@mail.gmail.com>
	<m2bov2ejd7.fsf@boostpro.com>
Message-ID: <CAPVmgLgepge5OPNgPXQzP47WU7mv+S5TKLQA1TS9t6kcBwqGOw@mail.gmail.com>

Here's a really short example of what I don't understand.  Basically I
can setup the python function to accept shared pointers, but can I
setup it to also use normal pointers?  In the example code attached,
if I uncomment scene->sendYourselfToPython(); it fails because it's
using "this" as the scene pointer.

Error in Python: <type 'exceptions.TypeError'>: No to_python
(by-value) converter found for C++ type: Scene

So it appears to only work for shared pointers at this point.  Can I
also get it to work with "this" at the same time?  Jim pointed me to
"enable_shared_from_this" as a way for creating a shared pointer
object, which makes sense from the docs of how to do it, but I was
wondering if I necessarily NEED to send it down as a shared pointer.

Thanks

http://dpaste.com/607969/

OR

#include <iostream>

#include <boost/python.hpp>
#include <boost/python/class.hpp>
#include <boost/python/module.hpp>
#include <boost/python/def.hpp>

using namespace boost::python;

object pyMainModule;
object pyMainNamespace;

#define EXAMPLE_PY_FUNCTION \
  "from scene import Scene\n" \
  "class Foo(object):\n" \
  "  @staticmethod\n" \
  "  def processFile(scene, filename):\n" \
  "    print('here')\n"

std::string parse_python_exception();

class Scene
{
public:
  void sendYourselfToPython()
  {
    try {
      object ignored = exec(EXAMPLE_PY_FUNCTION, pyMainNamespace);
      object processFileFunc = pyMainModule.attr("Foo").attr("processFile");
      processFileFunc(this, "test.txt");
    } catch (boost::python::error_already_set const &) {
      std::string perror = parse_python_exception();
      std::cerr << "Error in Python: " << perror << std::endl;
    }
  }
};
typedef boost::shared_ptr<Scene> SceneP;

BOOST_PYTHON_MODULE(scene)
{
  class_<Scene, boost::noncopyable>("Scene");
}

main(int argc, char**argv)
{
  std::cout << "starting program..." << std::endl;

  Py_Initialize();
  pyMainModule = import("__main__");
  pyMainNamespace = pyMainModule.attr("__dict__");

  boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
  PyImport_AppendInittab("scene", &initscene);

  SceneP scene(new Scene());
  scene->sendYourselfToPython(); // confused here

  try {
    object ignored = exec(EXAMPLE_PY_FUNCTION, pyMainNamespace);
    object processFileFunc = pyMainModule.attr("Foo").attr("processFile");
    processFileFunc(scene, "test.txt");
  } catch (boost::python::error_already_set const &) {
    std::string perror = parse_python_exception();
    std::cerr << "Error in Python: " << perror << std::endl;
  }
}


// taken from http://thejosephturner.com/blog/2011/06/15/embedding-python-in-c-applications-with-boostpython-part-2/
namespace py = boost::python;
std::string parse_python_exception() {
    PyObject *type_ptr = NULL, *value_ptr = NULL, *traceback_ptr = NULL;
    PyErr_Fetch(&type_ptr, &value_ptr, &traceback_ptr);
    std::string ret("Unfetchable Python error");
    if (type_ptr != NULL) {
        py::handle<> h_type(type_ptr);
        py::str type_pstr(h_type);
        py::extract<std::string> e_type_pstr(type_pstr);
        if(e_type_pstr.check())
            ret = e_type_pstr();
        else
            ret = "Unknown exception type";
    }

    if (value_ptr != NULL) {
        py::handle<> h_val(value_ptr);
        py::str a(h_val);
        py::extract<std::string> returned(a);
        if(returned.check())
            ret +=  ": " + returned();
        else
            ret += std::string(": Unparseable Python error: ");
    }

    if (traceback_ptr != NULL) {
        py::handle<> h_tb(traceback_ptr);
        py::object tb(py::import("traceback"));
        py::object fmt_tb(tb.attr("format_tb"));
        py::object tb_list(fmt_tb(h_tb));
        py::object tb_str(py::str("\n").join(tb_list));
        py::extract<std::string> returned(tb_str);
        if(returned.check())
            ret += ": " + returned();
        else
            ret += std::string(": Unparseable Python traceback");
    }
    return ret;
}


On Fri, Sep 2, 2011 at 2:15 PM, Dave Abrahams <dave at boostpro.com> wrote:
>
> on Fri Sep 02 2011, Josh Stratton <strattonbrazil-AT-gmail.com> wrote:
>
>> Well, right now I'm just passing "this" from inside the scene object,
>> so does that need to be wrapped in a shared pointer?
>
> That's a pretty vague description of what you're doing, so it's hard to
> say. ?I suggest you reduce your question to a minimal case that doesn't
> involve any of your domain-specific stuff. ?Then it will be easy to
> answer.
>
>>
>> On Fri, Sep 2, 2011 at 12:19 PM, Dave Abrahams <dave at boostpro.com> wrote:
>>>
>>> on Thu Sep 01 2011, Jim Bosch <talljimbo-AT-gmail.com> wrote:
>>>
>>>> boost::python::register_ptr_to_python< boost::shared_ptr<Scene> >();
>>>>
>>>> This allows you to return shared_ptr to Python, but because a
>>>> boost::shared_ptr can use an arbitrary deleter, Boost.Python can
>>>> convert any wrapped Scene object (even one that doesn't hold a
>>>> shared_ptr<Scene>) into a shared_ptr<Scene> with correct reference
>>>> counting.
>>>
>>> I might have forgotten by now, but I don't think you need to register
>>> shared_ptr, and I don't think the above will have any effect. ?IIRC I
>>> made sure shared_ptr support "just works" out-of-the-box.
>>>
>>> --
>>> Dave Abrahams
>>> BoostPro Computing
>>> http://www.boostpro.com>
>>> _______________________________________________
>>> Cplusplus-sig mailing list
>>> Cplusplus-sig at python.org
>>> http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From dave at boostpro.com  Sat Sep  3 06:08:40 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Fri, 02 Sep 2011 20:08:40 -0800
Subject: [C++-sig] copy constructors and multiple instances
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com> <m21uvyg3a9.fsf@boostpro.com>
	<CAPVmgLiGPf9S5uD3OEk0o3GtCyy5wmrGd-h-7_31iDCAVYVRRA@mail.gmail.com>
	<m2bov2ejd7.fsf@boostpro.com>
	<CAPVmgLgepge5OPNgPXQzP47WU7mv+S5TKLQA1TS9t6kcBwqGOw@mail.gmail.com>
Message-ID: <m2y5y68dyv.fsf@boostpro.com>


on Fri Sep 02 2011, Josh Stratton <strattonbrazil-AT-gmail.com> wrote:

> Here's a really short example of what I don't understand.  Basically I
> can setup the python function to accept shared pointers, but can I
> setup it to also use normal pointers?  

Yes, once you've exposed the class to Python, you can pass pointers to
Python.  However, Boost.Python won't let you pass raw pointers around
without telling it (via call policies) what you want to do about
lifetime management... actually what it will do, if you don't tell it
otherwise, is copy the pointee, which is at least safe from an object
lifetime point-of-view.

> In the example code attached, if I uncomment
> scene->sendYourselfToPython(); it fails because it's using "this" as
> the scene pointer.

(It's already uncommented)

> Error in Python: <type 'exceptions.TypeError'>: No to_python
> (by-value) converter found for C++ type: Scene

...because you've said it's noncopyable, there's no way to copy the
pointee, which is what happens when it is passed by value.

> So it appears to only work for shared pointers at this point.  Can I
> also get it to work with "this" at the same time?  Jim pointed me to
> "enable_shared_from_this" as a way for creating a shared pointer
> object, which makes sense from the docs of how to do it, but I was
> wondering if I necessarily NEED to send it down as a shared pointer.

There *are* things you can do to subvert Python's checking, but they're
highly discouraged.  Jim is on the right track: if you want to pass a
mutable reference to the object and you want its lifetime to be properly
managed, you should do what he's saying.  If you don't like
using enable_shared_from_this, of course, you can just do it outside a
member function:

  void sendSceneToPython(SceneP s)
    try {
      object ignored = exec(EXAMPLE_PY_FUNCTION, pyMainNamespace);
      object processFileFunc = pyMainModule.attr("Foo").attr("processFile");
      processFileFunc(s, "test.txt");
    } catch (boost::python::error_already_set const &) {
      std::string perror = parse_python_exception();
      std::cerr << "Error in Python: " << perror << std::endl;
    }
  }

HTH,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From ndbecker2 at gmail.com  Sat Sep  3 13:58:18 2011
From: ndbecker2 at gmail.com (Neal Becker)
Date: Sat, 03 Sep 2011 07:58:18 -0400
Subject: [C++-sig] pickle the unpickleable
Message-ID: <j3t4or$s3f$1@dough.gmane.org>

I have a pickling problem I suspect is just not possible to solve.

I have some wrappers around boost::random objects.

There is a rng object, which is a single underlying mersenne twister.

There are various distributions.  For example, uniform_real.

Each uniform_real holds a reference to the single, external mt object.

I don't think there's any way to unpickle several instance of uniform_real so 
that their references to the (shared) mt object get correctly restored.

Imagine something like this:

class mt;

struct uniform_real {
  mt & _mt;

  uniform_real (mt &, min, max) ...
};

When unpickled, the internal _mt states are saved and restored correctly, but 2 
instances of uniform_real will not share a reference to a single mt object, they 
each have their own copy which starts with a copy of the same state (no good).


From strattonbrazil at gmail.com  Sat Sep  3 16:40:47 2011
From: strattonbrazil at gmail.com (Josh Stratton)
Date: Sat, 3 Sep 2011 07:40:47 -0700
Subject: [C++-sig] copy constructors and multiple instances
In-Reply-To: <m2y5y68dyv.fsf@boostpro.com>
References: <CAPVmgLi4m25PjxvvFP8nVG50OzwBDCxHo0n2tTb-Q62J4SBH7g@mail.gmail.com>
	<4E5FCACD.6020008@gmail.com>
	<CAPVmgLjvzUxs9g8QtGDxYPfsBJRemVzpkMcsjf-yzw=xVE_Ukw@mail.gmail.com>
	<4E5FF0A8.1080004@gmail.com> <m21uvyg3a9.fsf@boostpro.com>
	<CAPVmgLiGPf9S5uD3OEk0o3GtCyy5wmrGd-h-7_31iDCAVYVRRA@mail.gmail.com>
	<m2bov2ejd7.fsf@boostpro.com>
	<CAPVmgLgepge5OPNgPXQzP47WU7mv+S5TKLQA1TS9t6kcBwqGOw@mail.gmail.com>
	<m2y5y68dyv.fsf@boostpro.com>
Message-ID: <CAPVmgLh09gmptwH1e7PN28bxpvFZZ3aq8F-Ea_8ArZ9Y+JbqbA@mail.gmail.com>

Thanks for everyone's help.  I'm posting my updated class that is now working.

class Scene : public boost::enable_shared_from_this<Scene>
{
public:
  void sendYourselfToPython()
  {
    try {
      object ignored = exec(EXAMPLE_PY_FUNCTION, pyMainNamespace);
      object processFileFunc = pyMainModule.attr("Foo").attr("processFile");
      processFileFunc(shared_from_this(), "test.txt");
    } catch (boost::python::error_already_set const &) {
      std::string perror = parse_python_exception();
      std::cerr << "Error in Python: " << perror << std::endl;
    }
  }
};
typedef boost::shared_ptr<Scene> SceneP;

On Fri, Sep 2, 2011 at 9:08 PM, Dave Abrahams <dave at boostpro.com> wrote:
>
> on Fri Sep 02 2011, Josh Stratton <strattonbrazil-AT-gmail.com> wrote:
>
>> Here's a really short example of what I don't understand. ?Basically I
>> can setup the python function to accept shared pointers, but can I
>> setup it to also use normal pointers?
>
> Yes, once you've exposed the class to Python, you can pass pointers to
> Python. ?However, Boost.Python won't let you pass raw pointers around
> without telling it (via call policies) what you want to do about
> lifetime management... actually what it will do, if you don't tell it
> otherwise, is copy the pointee, which is at least safe from an object
> lifetime point-of-view.
>
>> In the example code attached, if I uncomment
>> scene->sendYourselfToPython(); it fails because it's using "this" as
>> the scene pointer.
>
> (It's already uncommented)
>
>> Error in Python: <type 'exceptions.TypeError'>: No to_python
>> (by-value) converter found for C++ type: Scene
>
> ...because you've said it's noncopyable, there's no way to copy the
> pointee, which is what happens when it is passed by value.
>
>> So it appears to only work for shared pointers at this point. ?Can I
>> also get it to work with "this" at the same time? ?Jim pointed me to
>> "enable_shared_from_this" as a way for creating a shared pointer
>> object, which makes sense from the docs of how to do it, but I was
>> wondering if I necessarily NEED to send it down as a shared pointer.
>
> There *are* things you can do to subvert Python's checking, but they're
> highly discouraged. ?Jim is on the right track: if you want to pass a
> mutable reference to the object and you want its lifetime to be properly
> managed, you should do what he's saying. ?If you don't like
> using enable_shared_from_this, of course, you can just do it outside a
> member function:
>
> ?void sendSceneToPython(SceneP s)
> ? ?try {
> ? ? ?object ignored = exec(EXAMPLE_PY_FUNCTION, pyMainNamespace);
> ? ? ?object processFileFunc = pyMainModule.attr("Foo").attr("processFile");
> ? ? ?processFileFunc(s, "test.txt");
> ? ?} catch (boost::python::error_already_set const &) {
> ? ? ?std::string perror = parse_python_exception();
> ? ? ?std::cerr << "Error in Python: " << perror << std::endl;
> ? ?}
> ?}
>
> HTH,
>
> --
> Dave Abrahams
> BoostPro Computing
> http://www.boostpro.com
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From talljimbo at gmail.com  Mon Sep  5 21:31:30 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Mon, 05 Sep 2011 12:31:30 -0700
Subject: [C++-sig] pickle the unpickleable
In-Reply-To: <j3t4or$s3f$1@dough.gmane.org>
References: <j3t4or$s3f$1@dough.gmane.org>
Message-ID: <4E652392.9040800@gmail.com>

On 09/03/2011 04:58 AM, Neal Becker wrote:
> I have a pickling problem I suspect is just not possible to solve.
>
> I have some wrappers around boost::random objects.
>
> There is a rng object, which is a single underlying mersenne twister.
>
> There are various distributions.  For example, uniform_real.
>
> Each uniform_real holds a reference to the single, external mt object.
>
> I don't think there's any way to unpickle several instance of uniform_real so
> that their references to the (shared) mt object get correctly restored.
>
> Imagine something like this:
>
> class mt;
>
> struct uniform_real {
>    mt&  _mt;
>
>    uniform_real (mt&, min, max) ...
> };
>
> When unpickled, the internal _mt states are saved and restored correctly, but 2
> instances of uniform_real will not share a reference to a single mt object, they
> each have their own copy which starts with a copy of the same state (no good).
>

You might be able to get something working if you use Pickler and 
Unpickler objects instead of the convenience functions in 
pickle/cPickle.  Check out the "memo" and "persistent_id" attributes in 
particular (in the Python library reference docs).

Jim

From ndbecker2 at gmail.com  Tue Sep  6 19:14:30 2011
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 06 Sep 2011 13:14:30 -0400
Subject: [C++-sig] Solved (pickle multiple refs to objects)
References: <j3t4or$s3f$1@dough.gmane.org> <4E652392.9040800@gmail.com>
Message-ID: <j45kdn$ks$1@dough.gmane.org>

The solution to my problem is a combination of boost::serialization and 
boost::shared_ptr.

Rather than holding refs to the contained shared objects, if the objects are 
share_ptr, then serialization will do the correct thing.

For example, I have

struct boost_uniform_int_wrap {
  boost::shared_ptr<rng_t> rng_obj;
  boost::uniform_int<> gen;

  boost_uniform_int_wrap (boost::shared_ptr<rng_t> _rng_obj, int min, int max) :
    rng_obj (_rng_obj),
    gen (min, max) {}
...

struct uniform_int_pickle_suite : bp::pickle_suite {
  static bp::tuple getinitargs (boost_uniform_int_wrap const& gen) {
    return bp::make_tuple (gen.rng_obj, gen.gen.min(), gen.gen.max());
  }
};

Also, I have modified mersenne_twister to make it serializable (very simple to 
add).

Then to make mersenne_twister pickleable:

typedef boost::mt19937 rng_t;

struct mt_pickle_suite : bp::pickle_suite {
    
  static bp::object getstate (const rng_t& rng) {
    std::ostringstream os;
    boost::archive::binary_oarchive oa(os);
    oa << rng;
    return bp::str (os.str());
  }

  static void
  setstate(rng_t& rng, bp::object entries) {
    bp::str s = bp::extract<bp::str> (entries)();
    std::string st = bp::extract<std::string> (s)();
    std::istringstream is (st);
    
    boost::archive::binary_iarchive ia (is);
    ia >> rng;
  }
};

Now:
from boost_rand import rng, normal
from sys import getrefcount

rng1 = rng()
print getrefcount(rng1)
n1 = normal (rng1, 1, 0)
print getrefcount(rng1)
n2 = normal (rng1, 1, 0)
print getrefcount(rng1)

from cPickle import dumps, loads
rng2, n3, n4 = loads (dumps ((rng1, n1, n2), -1))

In [10]: ## working on region in file /usr/tmp/python-18983NT.py...
2
3
4

In [11]: rng2
Out[11]: <boost_rand.rng at 0x307c838>

In [12]: n3.rng
Out[12]: <boost_rand.rng at 0x307c838>

In [13]: n4.rng
Out[13]: <boost_rand.rng at 0x307c838>


From paul at kroitor.ca  Tue Sep  6 20:01:07 2011
From: paul at kroitor.ca (Paul Kroitor)
Date: Tue, 6 Sep 2011 14:01:07 -0400
Subject: [C++-sig] Build Errors Making the "quickstart" Demo Project
Message-ID: <A85E1A73F132DC46BEAFA87839227E15B50145@samwise.jil.local>

Hello all,

 

We are adding a python scripting layer on top of an existing largish C++
app, and are hoping to use Boost.Python to accomplish this. So far, it's
hasn't been going very smoothly, but I think we are nearly to the point
of getting the demo program to compile.

 

For the most part, we are following the instructions here:

http://www.boost.org/doc/libs/1_47_0/libs/python/doc/building.html

 

but have also had to use hints from here

http://stackoverflow.com/questions/2629421/how-to-use-boost-in-visual-st
udio-2010

and several other forums to get as far as we have (who / where should we
report manual errors to?).

 

I don't think we are doing anything non-standard - we certainly aren't
trying to - but perhaps some of the issues we are seeing come from using
Python 3.2. To complete the list, we are also using MSVC 10.0 SP1 and
Boost 1.47 on Windows 7 32-bit platform.

 

Currently, we are trying to get the "quickstart" example working and are
stuck on an error in the build of  "embedding.cpp":

 

(note that for clarity I've rerun the build a second time so that it is
only trying to build the failed modules)

 

........................................................................
...............

 

C:\Boost\boost_1_47_0\libs\python\example\quickstart>b2

...patience...

...patience...

...found 1603 targets...

...updating 3 targets...

compile-c-c++ bin\msvc-10.0\debug\threading-multi\embedding.obj

embedding.cpp

embedding.cpp(39) : error C2668:
'std::basic_string<_Elem,_Traits,_Ax>::basic_st

ring' : ambiguous call to overloaded function

        with

        [

            _Elem=char,

            _Traits=std::char_traits<char>,

            _Ax=std::allocator<char>

        ]

        c:\Program Files\Microsoft Visual Studio
10.0\VC\INCLUDE\xstring(700): c

ould be
'std::basic_string<_Elem,_Traits,_Ax>::basic_string(std::basic_string<_E

lem,_Traits,_Ax> &&)'

        with

        [

            _Elem=char,

            _Traits=std::char_traits<char>,

            _Ax=std::allocator<char>

        ]

        c:\Program Files\Microsoft Visual Studio
10.0\VC\INCLUDE\xstring(590): o

r       'std::basic_string<_Elem,_Traits,_Ax>::basic_string(const _Elem
*)'

        with

        [

            _Elem=char,

            _Traits=std::char_traits<char>,

            _Ax=std::allocator<char>

        ]

        while trying to match the argument list
'(boost::python::detail::method_

result)'

embedding.cpp(56) : error C2065: 'initembedded_hello' : undeclared
identifier

 

    call "c:\Program Files\Microsoft Visual Studio
10.0\VC\vcvarsall.bat" x86 >n

ul

cl /Zm800 -nologo
@"bin\msvc-10.0\debug\threading-multi\embedding.obj.rsp"

 

...failed compile-c-c++
bin\msvc-10.0\debug\threading-multi\embedding.obj...

...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.exe for lack
of <pbin

\msvc-10.0\debug\threading-multi>embedding.obj...

...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.pdb for lack
of <pbin

\msvc-10.0\debug\threading-multi>embedding.obj...

...failed updating 1 target...

...skipped 2 targets...

 

........................................................................
...............

 

With all the templates and overloads, this is getting too technical for
to solve here. I am hoping someone here will recognize this error or
understand what's causing it.

 

Thanks in advance for any help or guidance anyone is able to give.

Paul Kroitor 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110906/4c8a91c2/attachment.html>

From s_sourceforge at nedprod.com  Wed Sep  7 12:57:19 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Wed, 07 Sep 2011 11:57:19 +0100
Subject: [C++-sig] Build Errors Making the "quickstart" Demo Project
In-Reply-To: <A85E1A73F132DC46BEAFA87839227E15B50145@samwise.jil.local>
References: <A85E1A73F132DC46BEAFA87839227E15B50145@samwise.jil.local>
Message-ID: <4E674E0F.5158.6CF46FA0@s_sourceforge.nedprod.com>

If I am understanding you correctly, your attempts to build and/or 
use stock BPL is failing when building the quickstart demo?

If so, there is surely some issue with your build environment as 
Boost runs a series of automated regression tests which catch build 
failures.

If I'm not right in understanding you correctly, please do explain 
further.

Niall


On 6 Sep 2011 at 14:01, Paul Kroitor wrote:

> Hello all,
> 
>  
> 
> We are adding a python scripting layer on top of an existing largish C++
> app, and are hoping to use Boost.Python to accomplish this. So far, it's
> hasn't been going very smoothly, but I think we are nearly to the point
> of getting the demo program to compile.
> 
>  
> 
> For the most part, we are following the instructions here:
> 
> http://www.boost.org/doc/libs/1_47_0/libs/python/doc/building.html
> 
>  
> 
> but have also had to use hints from here
> 
> http://stackoverflow.com/questions/2629421/how-to-use-boost-in-visual-st
> udio-2010
> 
> and several other forums to get as far as we have (who / where should we
> report manual errors to?).
> 
>  
> 
> I don't think we are doing anything non-standard - we certainly aren't
> trying to - but perhaps some of the issues we are seeing come from using
> Python 3.2. To complete the list, we are also using MSVC 10.0 SP1 and
> Boost 1.47 on Windows 7 32-bit platform.
> 
>  
> 
> Currently, we are trying to get the "quickstart" example working and are
> stuck on an error in the build of  "embedding.cpp":
> 
>  
> 
> (note that for clarity I've rerun the build a second time so that it is
> only trying to build the failed modules)
> 
>  
> 
> ........................................................................
> ...............
> 
>  
> 
> C:\Boost\boost_1_47_0\libs\python\example\quickstart>b2
> 
> ...patience...
> 
> ...patience...
> 
> ...found 1603 targets...
> 
> ...updating 3 targets...
> 
> compile-c-c++ bin\msvc-10.0\debug\threading-multi\embedding.obj
> 
> embedding.cpp
> 
> embedding.cpp(39) : error C2668:
> 'std::basic_string<_Elem,_Traits,_Ax>::basic_st
> 
> ring' : ambiguous call to overloaded function
> 
>         with
> 
>         [
> 
>             _Elem=char,
> 
>             _Traits=std::char_traits<char>,
> 
>             _Ax=std::allocator<char>
> 
>         ]
> 
>         c:\Program Files\Microsoft Visual Studio
> 10.0\VC\INCLUDE\xstring(700): c
> 
> ould be
> 'std::basic_string<_Elem,_Traits,_Ax>::basic_string(std::basic_string<_E
> 
> lem,_Traits,_Ax> &&)'
> 
>         with
> 
>         [
> 
>             _Elem=char,
> 
>             _Traits=std::char_traits<char>,
> 
>             _Ax=std::allocator<char>
> 
>         ]
> 
>         c:\Program Files\Microsoft Visual Studio
> 10.0\VC\INCLUDE\xstring(590): o
> 
> r       'std::basic_string<_Elem,_Traits,_Ax>::basic_string(const _Elem
> *)'
> 
>         with
> 
>         [
> 
>             _Elem=char,
> 
>             _Traits=std::char_traits<char>,
> 
>             _Ax=std::allocator<char>
> 
>         ]
> 
>         while trying to match the argument list
> '(boost::python::detail::method_
> 
> result)'
> 
> embedding.cpp(56) : error C2065: 'initembedded_hello' : undeclared
> identifier
> 
>  
> 
>     call "c:\Program Files\Microsoft Visual Studio
> 10.0\VC\vcvarsall.bat" x86 >n
> 
> ul
> 
> cl /Zm800 -nologo
> @"bin\msvc-10.0\debug\threading-multi\embedding.obj.rsp"
> 
>  
> 
> ...failed compile-c-c++
> bin\msvc-10.0\debug\threading-multi\embedding.obj...
> 
> ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.exe for lack
> of <pbin
> 
> \msvc-10.0\debug\threading-multi>embedding.obj...
> 
> ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.pdb for lack
> of <pbin
> 
> \msvc-10.0\debug\threading-multi>embedding.obj...
> 
> ...failed updating 1 target...
> 
> ...skipped 2 targets...
> 
>  
> 
> ........................................................................
> ...............
> 
>  
> 
> With all the templates and overloads, this is getting too technical for
> to solve here. I am hoping someone here will recognize this error or
> understand what's causing it.
> 
>  
> 
> Thanks in advance for any help or guidance anyone is able to give.
> 
> Paul Kroitor 
> 
> 


-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From paul at kroitor.ca  Wed Sep  7 18:11:23 2011
From: paul at kroitor.ca (Paul Kroitor)
Date: Wed, 7 Sep 2011 12:11:23 -0400
Subject: [C++-sig] Build Errors Making the "quickstart" Demo Project
Message-ID: <A85E1A73F132DC46BEAFA87839227E15B50179@samwise.jil.local>

Hi, thanks for your reply.

 

Yes, you are correct that "our attempts to build and/or use stock BPL is
failing when building the quickstart demo". The boost build completes
normally (but doesn't show any messages about running any tests) and
specifies strings to add to the LIB and INCLUDE paths. At that point we
simply CD to the quickstart sample directory and run "B2". This second
build fails with a compiler error about an ambiguous call to an
overloaded function.

 

Is there something else that needs to be set? My understanding was that
the earlier instructions for setting up the environment (bootstrap,
user-config.jam) set up the environment for building both boost (which
worked) and quickstart (which didn't).

 

Here is the complete sequence:

............................................

C:\Boost\boost_1_47_0>.\b2

 

Building the Boost C++ Libraries.

 

 

Performing configuration checks

 

    - has_icu builds           : no

warning: Graph library does not contain MPI-based parallel components.

note: to enable them, add "using mpi ;" to your user-config.jam

    - ../config//has_gcc_visibility builds : no

    - ../config//has_long_double_support builds : yes

warning: skipping optional Message Passing Interface (MPI) library.

note: to enable MPI support, add "using mpi ;" to user-config.jam.

note: to suppress this message, pass "--without-mpi" to bjam.

note: otherwise, you can safely ignore this message.

 

Component configuration:

 

    - chrono                   : building

    - date_time                : building

    - exception                : building

    - filesystem               : building

    - graph                    : building

    - graph_parallel           : building

    - iostreams                : building

    - math                     : building

    - mpi                      : building

    - program_options          : building

    - python                   : building

    - random                   : building

    - regex                    : building

    - serialization            : building

    - signals                  : building

    - system                   : building

    - test                     : building

    - thread                   : building

    - wave                     : building

 

...patience...

...patience...

...patience...

...patience...

...found 7688 targets...

 

 

The Boost C++ Libraries were successfully built!

 

The following directory should be added to compiler include paths:

 

    C:/Boost/boost_1_47_0

 

The following directory should be added to linker library paths:

 

    C:\Boost\boost_1_47_0\stage\lib

 

 

C:\Boost\boost_1_47_0>set include

Environment variable include not defined

 

C:\Boost\boost_1_47_0>set include=C:\Boost\Boost_1_47_0

 

C:\Boost\boost_1_47_0>set lib

Environment variable lib not defined

 

C:\Boost\boost_1_47_0>set lib=C:\Boost\boost_1_47_0\stage\lib

 

C:\Boost\boost_1_47_0>cd libs\python\example\quickstart

 

C:\Boost\boost_1_47_0\libs\python\example\quickstart>path
%path%;c:\boost\boost_

1_47_0

 

C:\Boost\boost_1_47_0\libs\python\example\quickstart>b2

...patience...

...patience...

...found 1603 targets...

...updating 3 targets...

compile-c-c++ bin\msvc-10.0\debug\threading-multi\embedding.obj

embedding.cpp

embedding.cpp(39) : error C2668:
'std::basic_string<_Elem,_Traits,_Ax>::basic_st

ring' : ambiguous call to overloaded function

        with

        [

            _Elem=char,

            _Traits=std::char_traits<char>,

            _Ax=std::allocator<char>

        ]

        c:\Program Files\Microsoft Visual Studio
10.0\VC\INCLUDE\xstring(700): c

ould be
'std::basic_string<_Elem,_Traits,_Ax>::basic_string(std::basic_string<_E

lem,_Traits,_Ax> &&)'

        with

        [

            _Elem=char,

            _Traits=std::char_traits<char>,

            _Ax=std::allocator<char>

        ]

        c:\Program Files\Microsoft Visual Studio
10.0\VC\INCLUDE\xstring(590): o

r       'std::basic_string<_Elem,_Traits,_Ax>::basic_string(const _Elem
*)'

        with

        [

            _Elem=char,

            _Traits=std::char_traits<char>,

            _Ax=std::allocator<char>

        ]

        while trying to match the argument list
'(boost::python::detail::method_

result)'

embedding.cpp(56) : error C2065: 'initembedded_hello' : undeclared
identifier

 

    call "c:\Program Files\Microsoft Visual Studio
10.0\VC\vcvarsall.bat" x86 >n

ul

cl /Zm800 -nologo
@"bin\msvc-10.0\debug\threading-multi\embedding.obj.rsp"

 

...failed compile-c-c++
bin\msvc-10.0\debug\threading-multi\embedding.obj...

...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.exe for lack
of <pbin

\msvc-10.0\debug\threading-multi>embedding.obj...

...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.pdb for lack
of <pbin

\msvc-10.0\debug\threading-multi>embedding.obj...

...failed updating 1 target...

...skipped 2 targets...

......................................

 

 

 

 

Am I missing something?

 

 

 

_____________________________________________________

 

If I am understanding you correctly, your attempts to build and/or use
stock BPL is failing when building the quickstart demo?

 

If so, there is surely some issue with your build environment as Boost
runs a series of automated regression tests which catch build failures.

 

If I'm not right in understanding you correctly, please do explain
further.

 

Niall

 

 

On 6 Sep 2011 at 14:01, Paul Kroitor wrote:

 

> Hello all,

> 

>  

> 

> We are adding a python scripting layer on top of an existing largish 

> C++ app, and are hoping to use Boost.Python to accomplish this. So 

> far, it's hasn't been going very smoothly, but I think we are nearly 

> to the point of getting the demo program to compile.

> 

>  

> 

> For the most part, we are following the instructions here:

> 

> http://www.boost.org/doc/libs/1_47_0/libs/python/doc/building.html

> 

>  

> 

> but have also had to use hints from here

> 

> http://stackoverflow.com/questions/2629421/how-to-use-boost-in-visual-

> st

> udio-2010

> 

> and several other forums to get as far as we have (who / where should 

> we report manual errors to?).

> 

>  

> 

> I don't think we are doing anything non-standard - we certainly aren't


> trying to - but perhaps some of the issues we are seeing come from 

> using Python 3.2. To complete the list, we are also using MSVC 10.0 

> SP1 and Boost 1.47 on Windows 7 32-bit platform.

> 

>  

> 

> Currently, we are trying to get the "quickstart" example working and 

> are stuck on an error in the build of  "embedding.cpp":

> 

>  

> 

> (note that for clarity I've rerun the build a second time so that it 

> is only trying to build the failed modules)

> 

>  

> 

>
........................................................................

> ...............

> 

>  

> 

> C:\Boost\boost_1_47_0\libs\python\example\quickstart>b2

> 

> ...patience...

> 

> ...patience...

> 

> ...found 1603 targets...

> 

> ...updating 3 targets...

> 

> compile-c-c++ bin\msvc-10.0\debug\threading-multi\embedding.obj

> 

> embedding.cpp

> 

> embedding.cpp(39) : error C2668:

> 'std::basic_string<_Elem,_Traits,_Ax>::basic_st

> 

> ring' : ambiguous call to overloaded function

> 

>         with

> 

>         [

> 

>             _Elem=char,

> 

>             _Traits=std::char_traits<char>,

> 

>             _Ax=std::allocator<char>

> 

>         ]

> 

>         c:\Program Files\Microsoft Visual Studio

> 10.0\VC\INCLUDE\xstring(700): c

> 

> ould be

> 'std::basic_string<_Elem,_Traits,_Ax>::basic_string(std::basic_string<

> _E

> 

> lem,_Traits,_Ax> &&)'

> 

>         with

> 

>         [

> 

>             _Elem=char,

> 

>             _Traits=std::char_traits<char>,

> 

>             _Ax=std::allocator<char>

> 

>         ]

> 

>         c:\Program Files\Microsoft Visual Studio

> 10.0\VC\INCLUDE\xstring(590): o

> 

> r       'std::basic_string<_Elem,_Traits,_Ax>::basic_string(const
_Elem

> *)'

> 

>         with

> 

>         [

> 

>             _Elem=char,

> 

>             _Traits=std::char_traits<char>,

> 

>             _Ax=std::allocator<char>

> 

>         ]

> 

>         while trying to match the argument list 

> '(boost::python::detail::method_

> 

> result)'

> 

> embedding.cpp(56) : error C2065: 'initembedded_hello' : undeclared 

> identifier

> 

>  

> 

>     call "c:\Program Files\Microsoft Visual Studio 

> 10.0\VC\vcvarsall.bat" x86 >n

> 

> ul

> 

> cl /Zm800 -nologo

> @"bin\msvc-10.0\debug\threading-multi\embedding.obj.rsp"

> 

>  

> 

> ...failed compile-c-c++

> bin\msvc-10.0\debug\threading-multi\embedding.obj...

> 

> ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.exe for 

> lack of <pbin

> 

> \msvc-10.0\debug\threading-multi>embedding.obj...

> 

> ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.pdb for 

> lack of <pbin

> 

> \msvc-10.0\debug\threading-multi>embedding.obj...

> 

> ...failed updating 1 target...

> 

> ...skipped 2 targets...

> 

>  

> 

>
........................................................................

> ...............

> 

>  

> 

> With all the templates and overloads, this is getting too technical 

> for to solve here. I am hoping someone here will recognize this error 

> or understand what's causing it.

> 

>  

> 

> Thanks in advance for any help or guidance anyone is able to give.

> 

> Paul Kroitor

> 

> 

 

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110907/bc8505e3/attachment-0001.html>

From brandsmeier at gmx.de  Thu Sep  8 11:56:23 2011
From: brandsmeier at gmx.de (Holger Brandsmeier)
Date: Thu, 8 Sep 2011 11:56:23 +0200
Subject: [C++-sig] Dynamic Cast failed over multiple modules
Message-ID: <CAFP7zVvx92zxvqStF6fGesyhd-SzNhb=JGpzoe5Wzx06Kgjyxg@mail.gmail.com>

Dear List,

when I use dynamic cast from two boost modules on a class with "vague
linkage", in my real use case a template in the simplified example I
show you a class which is declared inline, then the dynamic cast fails
in the second module.

So when I call the code from python from the first module (in which
the Vertex object has been created), then the dynamic cast succeeds:

"calling performTest(m) ...
I am a Vertex"

but when I do the same from the other module, then I get

"calling performTest2(m) ...
dynamic cast failed, I got the object:
I am a Vertex"

I attached a very simple example in the files:
  bugDynamicCastSimplified.h (declares the Classes VerboseObject,
Vertex and Mesh)
  bugDynamicCastPy.cpp (first boost module)
  bugDynamicCast2Py.cpp (second boost module)

The problem dissapears without the vague linkage, e.g. when you define
REMOVE_INLINE_DEFINITION in the code. This is not a very nice  option
for me, as in my real use case Vertex is a template, and this would
force me to initialize it for each type.

My gcc is:
> g++ --version
g++ (Gentoo 4.4.5 p1.2, pie-0.4.5) 4.4.5

And I am building the project with cmake (not boost). I read the FAQ
entry at http://gcc.gnu.org/faq.html#dso and added "-Wl,-E" to
CMAKE_SHARED_LINKER_FLAGS. About the RTLD_GLOBAL flag for dlopen, I
don't know how python handles this.

Do you have any ideas how to solve this? Or is this simply not
possible with python?

-Holger
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bugDynamicCastSimplified.h
Type: text/x-chdr
Size: 913 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110908/8fffad6a/attachment.h>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bugDynamicCastPy.cpp
Type: text/x-c++src
Size: 250 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110908/8fffad6a/attachment.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bugDynamicCast2Py.cpp
Type: text/x-c++src
Size: 213 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110908/8fffad6a/attachment-0001.cpp>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bugDynamicCastSimplified.cpp
Type: text/x-c++src
Size: 154 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110908/8fffad6a/attachment-0002.cpp>

From mathieu.malaterre at gmail.com  Thu Sep  8 12:00:06 2011
From: mathieu.malaterre at gmail.com (Mathieu Malaterre)
Date: Thu, 8 Sep 2011 12:00:06 +0200
Subject: [C++-sig] Dynamic Cast failed over multiple modules
In-Reply-To: <CAFP7zVvx92zxvqStF6fGesyhd-SzNhb=JGpzoe5Wzx06Kgjyxg@mail.gmail.com>
References: <CAFP7zVvx92zxvqStF6fGesyhd-SzNhb=JGpzoe5Wzx06Kgjyxg@mail.gmail.com>
Message-ID: <CA+7wUswJHYcCWQkb2-F7EtaRb13QkgqLrBvG7J1mEEwtp1DgNg@mail.gmail.com>

On Thu, Sep 8, 2011 at 11:56 AM, Holger Brandsmeier <brandsmeier at gmx.de> wrote:
[...]
> CMAKE_SHARED_LINKER_FLAGS. About the RTLD_GLOBAL flag for dlopen, I
> don't know how python handles this.
>
> Do you have any ideas how to solve this? Or is this simply not
> possible with python?

Here is what I use:

if os.name == 'posix':
  orig_dlopen_flags = sys.getdlopenflags()
  try:
    import dl
  except ImportError:
    try:
      import DLFCN as dl
    except ImportError:
      dl = None
  if dl:
    sys.setdlopenflags(dl.RTLD_NOW|dl.RTLD_GLOBAL)
  # now load the module !
  from gdcmswig import *
  sys.setdlopenflags(orig_dlopen_flags)
  del dl
  del orig_dlopen_flags


ref:
http://gdcm.git.sourceforge.net/git/gitweb.cgi?p=gdcm/gdcm;a=blob;f=Wrapping/Python/gdcm.py;h=fd7ee0da80297d608c2939b8661ce0e81c33314a;hb=HEAD

-- 
Mathieu

From s_sourceforge at nedprod.com  Thu Sep  8 14:12:50 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Thu, 08 Sep 2011 13:12:50 +0100
Subject: [C++-sig] Build Errors Making the "quickstart" Demo Project
In-Reply-To: <A85E1A73F132DC46BEAFA87839227E15B50179@samwise.jil.local>
References: <A85E1A73F132DC46BEAFA87839227E15B50179@samwise.jil.local>
Message-ID: <4E68B142.20879.725FEDDE@s_sourceforge.nedprod.com>

If you're building stock using bjam, and it's failing, it's a bug :)

See if there's anything similar at 
https://svn.boost.org/trac/boost/search?q=python&noquickjump=1&ticket=
on and if not, report it.

Sorry can't help much more personally. I haven't used BPL, or Boost 
for that matter, in over four years. Oddly most of my apps 
development is outside C++ nowadays in various interpreted languages, 
or plain old C weirdly.

Niall


On 7 Sep 2011 at 12:11, Paul Kroitor wrote:

> Hi, thanks for your reply.
> 
>  
> 
> Yes, you are correct that "our attempts to build and/or use stock BPL is
> failing when building the quickstart demo". The boost build completes
> normally (but doesn't show any messages about running any tests) and
> specifies strings to add to the LIB and INCLUDE paths. At that point we
> simply CD to the quickstart sample directory and run "B2". This second
> build fails with a compiler error about an ambiguous call to an
> overloaded function.
> 
>  
> 
> Is there something else that needs to be set? My understanding was that
> the earlier instructions for setting up the environment (bootstrap,
> user-config.jam) set up the environment for building both boost (which
> worked) and quickstart (which didn't).
> 
>  
> 
> Here is the complete sequence:
> 
> ............................................
> 
> C:\Boost\boost_1_47_0>.\b2
> 
>  
> 
> Building the Boost C++ Libraries.
> 
>  
> 
>  
> 
> Performing configuration checks
> 
>  
> 
>     - has_icu builds           : no
> 
> warning: Graph library does not contain MPI-based parallel components.
> 
> note: to enable them, add "using mpi ;" to your user-config.jam
> 
>     - ../config//has_gcc_visibility builds : no
> 
>     - ../config//has_long_double_support builds : yes
> 
> warning: skipping optional Message Passing Interface (MPI) library.
> 
> note: to enable MPI support, add "using mpi ;" to user-config.jam.
> 
> note: to suppress this message, pass "--without-mpi" to bjam.
> 
> note: otherwise, you can safely ignore this message.
> 
>  
> 
> Component configuration:
> 
>  
> 
>     - chrono                   : building
> 
>     - date_time                : building
> 
>     - exception                : building
> 
>     - filesystem               : building
> 
>     - graph                    : building
> 
>     - graph_parallel           : building
> 
>     - iostreams                : building
> 
>     - math                     : building
> 
>     - mpi                      : building
> 
>     - program_options          : building
> 
>     - python                   : building
> 
>     - random                   : building
> 
>     - regex                    : building
> 
>     - serialization            : building
> 
>     - signals                  : building
> 
>     - system                   : building
> 
>     - test                     : building
> 
>     - thread                   : building
> 
>     - wave                     : building
> 
>  
> 
> ...patience...
> 
> ...patience...
> 
> ...patience...
> 
> ...patience...
> 
> ...found 7688 targets...
> 
>  
> 
>  
> 
> The Boost C++ Libraries were successfully built!
> 
>  
> 
> The following directory should be added to compiler include paths:
> 
>  
> 
>     C:/Boost/boost_1_47_0
> 
>  
> 
> The following directory should be added to linker library paths:
> 
>  
> 
>     C:\Boost\boost_1_47_0\stage\lib
> 
>  
> 
>  
> 
> C:\Boost\boost_1_47_0>set include
> 
> Environment variable include not defined
> 
>  
> 
> C:\Boost\boost_1_47_0>set include=C:\Boost\Boost_1_47_0
> 
>  
> 
> C:\Boost\boost_1_47_0>set lib
> 
> Environment variable lib not defined
> 
>  
> 
> C:\Boost\boost_1_47_0>set lib=C:\Boost\boost_1_47_0\stage\lib
> 
>  
> 
> C:\Boost\boost_1_47_0>cd libs\python\example\quickstart
> 
>  
> 
> C:\Boost\boost_1_47_0\libs\python\example\quickstart>path
> %path%;c:\boost\boost_
> 
> 1_47_0
> 
>  
> 
> C:\Boost\boost_1_47_0\libs\python\example\quickstart>b2
> 
> ...patience...
> 
> ...patience...
> 
> ...found 1603 targets...
> 
> ...updating 3 targets...
> 
> compile-c-c++ bin\msvc-10.0\debug\threading-multi\embedding.obj
> 
> embedding.cpp
> 
> embedding.cpp(39) : error C2668:
> 'std::basic_string<_Elem,_Traits,_Ax>::basic_st
> 
> ring' : ambiguous call to overloaded function
> 
>         with
> 
>         [
> 
>             _Elem=char,
> 
>             _Traits=std::char_traits<char>,
> 
>             _Ax=std::allocator<char>
> 
>         ]
> 
>         c:\Program Files\Microsoft Visual Studio
> 10.0\VC\INCLUDE\xstring(700): c
> 
> ould be
> 'std::basic_string<_Elem,_Traits,_Ax>::basic_string(std::basic_string<_E
> 
> lem,_Traits,_Ax> &&)'
> 
>         with
> 
>         [
> 
>             _Elem=char,
> 
>             _Traits=std::char_traits<char>,
> 
>             _Ax=std::allocator<char>
> 
>         ]
> 
>         c:\Program Files\Microsoft Visual Studio
> 10.0\VC\INCLUDE\xstring(590): o
> 
> r       'std::basic_string<_Elem,_Traits,_Ax>::basic_string(const _Elem
> *)'
> 
>         with
> 
>         [
> 
>             _Elem=char,
> 
>             _Traits=std::char_traits<char>,
> 
>             _Ax=std::allocator<char>
> 
>         ]
> 
>         while trying to match the argument list
> '(boost::python::detail::method_
> 
> result)'
> 
> embedding.cpp(56) : error C2065: 'initembedded_hello' : undeclared
> identifier
> 
>  
> 
>     call "c:\Program Files\Microsoft Visual Studio
> 10.0\VC\vcvarsall.bat" x86 >n
> 
> ul
> 
> cl /Zm800 -nologo
> @"bin\msvc-10.0\debug\threading-multi\embedding.obj.rsp"
> 
>  
> 
> ...failed compile-c-c++
> bin\msvc-10.0\debug\threading-multi\embedding.obj...
> 
> ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.exe for lack
> of <pbin
> 
> \msvc-10.0\debug\threading-multi>embedding.obj...
> 
> ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.pdb for lack
> of <pbin
> 
> \msvc-10.0\debug\threading-multi>embedding.obj...
> 
> ...failed updating 1 target...
> 
> ...skipped 2 targets...
> 
> ......................................
> 
>  
> 
>  
> 
>  
> 
>  
> 
> Am I missing something?
> 
>  
> 
>  
> 
>  
> 
> _____________________________________________________
> 
>  
> 
> If I am understanding you correctly, your attempts to build and/or use
> stock BPL is failing when building the quickstart demo?
> 
>  
> 
> If so, there is surely some issue with your build environment as Boost
> runs a series of automated regression tests which catch build failures.
> 
>  
> 
> If I'm not right in understanding you correctly, please do explain
> further.
> 
>  
> 
> Niall
> 
>  
> 
>  
> 
> On 6 Sep 2011 at 14:01, Paul Kroitor wrote:
> 
>  
> 
> > Hello all,
> 
> > 
> 
> >  
> 
> > 
> 
> > We are adding a python scripting layer on top of an existing largish 
> 
> > C++ app, and are hoping to use Boost.Python to accomplish this. So 
> 
> > far, it's hasn't been going very smoothly, but I think we are nearly 
> 
> > to the point of getting the demo program to compile.
> 
> > 
> 
> >  
> 
> > 
> 
> > For the most part, we are following the instructions here:
> 
> > 
> 
> > http://www.boost.org/doc/libs/1_47_0/libs/python/doc/building.html
> 
> > 
> 
> >  
> 
> > 
> 
> > but have also had to use hints from here
> 
> > 
> 
> > http://stackoverflow.com/questions/2629421/how-to-use-boost-in-visual-
> 
> > st
> 
> > udio-2010
> 
> > 
> 
> > and several other forums to get as far as we have (who / where should 
> 
> > we report manual errors to?).
> 
> > 
> 
> >  
> 
> > 
> 
> > I don't think we are doing anything non-standard - we certainly aren't
> 
> 
> > trying to - but perhaps some of the issues we are seeing come from 
> 
> > using Python 3.2. To complete the list, we are also using MSVC 10.0 
> 
> > SP1 and Boost 1.47 on Windows 7 32-bit platform.
> 
> > 
> 
> >  
> 
> > 
> 
> > Currently, we are trying to get the "quickstart" example working and 
> 
> > are stuck on an error in the build of  "embedding.cpp":
> 
> > 
> 
> >  
> 
> > 
> 
> > (note that for clarity I've rerun the build a second time so that it 
> 
> > is only trying to build the failed modules)
> 
> > 
> 
> >  
> 
> > 
> 
> >
> ........................................................................
> 
> > ...............
> 
> > 
> 
> >  
> 
> > 
> 
> > C:\Boost\boost_1_47_0\libs\python\example\quickstart>b2
> 
> > 
> 
> > ...patience...
> 
> > 
> 
> > ...patience...
> 
> > 
> 
> > ...found 1603 targets...
> 
> > 
> 
> > ...updating 3 targets...
> 
> > 
> 
> > compile-c-c++ bin\msvc-10.0\debug\threading-multi\embedding.obj
> 
> > 
> 
> > embedding.cpp
> 
> > 
> 
> > embedding.cpp(39) : error C2668:
> 
> > 'std::basic_string<_Elem,_Traits,_Ax>::basic_st
> 
> > 
> 
> > ring' : ambiguous call to overloaded function
> 
> > 
> 
> >         with
> 
> > 
> 
> >         [
> 
> > 
> 
> >             _Elem=char,
> 
> > 
> 
> >             _Traits=std::char_traits<char>,
> 
> > 
> 
> >             _Ax=std::allocator<char>
> 
> > 
> 
> >         ]
> 
> > 
> 
> >         c:\Program Files\Microsoft Visual Studio
> 
> > 10.0\VC\INCLUDE\xstring(700): c
> 
> > 
> 
> > ould be
> 
> > 'std::basic_string<_Elem,_Traits,_Ax>::basic_string(std::basic_string<
> 
> > _E
> 
> > 
> 
> > lem,_Traits,_Ax> &&)'
> 
> > 
> 
> >         with
> 
> > 
> 
> >         [
> 
> > 
> 
> >             _Elem=char,
> 
> > 
> 
> >             _Traits=std::char_traits<char>,
> 
> > 
> 
> >             _Ax=std::allocator<char>
> 
> > 
> 
> >         ]
> 
> > 
> 
> >         c:\Program Files\Microsoft Visual Studio
> 
> > 10.0\VC\INCLUDE\xstring(590): o
> 
> > 
> 
> > r       'std::basic_string<_Elem,_Traits,_Ax>::basic_string(const
> _Elem
> 
> > *)'
> 
> > 
> 
> >         with
> 
> > 
> 
> >         [
> 
> > 
> 
> >             _Elem=char,
> 
> > 
> 
> >             _Traits=std::char_traits<char>,
> 
> > 
> 
> >             _Ax=std::allocator<char>
> 
> > 
> 
> >         ]
> 
> > 
> 
> >         while trying to match the argument list 
> 
> > '(boost::python::detail::method_
> 
> > 
> 
> > result)'
> 
> > 
> 
> > embedding.cpp(56) : error C2065: 'initembedded_hello' : undeclared 
> 
> > identifier
> 
> > 
> 
> >  
> 
> > 
> 
> >     call "c:\Program Files\Microsoft Visual Studio 
> 
> > 10.0\VC\vcvarsall.bat" x86 >n
> 
> > 
> 
> > ul
> 
> > 
> 
> > cl /Zm800 -nologo
> 
> > @"bin\msvc-10.0\debug\threading-multi\embedding.obj.rsp"
> 
> > 
> 
> >  
> 
> > 
> 
> > ...failed compile-c-c++
> 
> > bin\msvc-10.0\debug\threading-multi\embedding.obj...
> 
> > 
> 
> > ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.exe for 
> 
> > lack of <pbin
> 
> > 
> 
> > \msvc-10.0\debug\threading-multi>embedding.obj...
> 
> > 
> 
> > ...skipped <pbin\msvc-10.0\debug\threading-multi>embedding.pdb for 
> 
> > lack of <pbin
> 
> > 
> 
> > \msvc-10.0\debug\threading-multi>embedding.obj...
> 
> > 
> 
> > ...failed updating 1 target...
> 
> > 
> 
> > ...skipped 2 targets...
> 
> > 
> 
> >  
> 
> > 
> 
> >
> ........................................................................
> 
> > ...............
> 
> > 
> 
> >  
> 
> > 
> 
> > With all the templates and overloads, this is getting too technical 
> 
> > for to solve here. I am hoping someone here will recognize this error 
> 
> > or understand what's causing it.
> 
> > 
> 
> >  
> 
> > 
> 
> > Thanks in advance for any help or guidance anyone is able to give.
> 
> > 
> 
> > Paul Kroitor
> 
> > 
> 
> > 
> 
>  
> 
>  
> 
> 


-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From brandsmeier at gmx.de  Thu Sep  8 14:14:38 2011
From: brandsmeier at gmx.de (Holger Brandsmeier)
Date: Thu, 8 Sep 2011 14:14:38 +0200
Subject: [C++-sig] Dynamic Cast failed over multiple modules
In-Reply-To: <CA+7wUswJHYcCWQkb2-F7EtaRb13QkgqLrBvG7J1mEEwtp1DgNg@mail.gmail.com>
References: <CAFP7zVvx92zxvqStF6fGesyhd-SzNhb=JGpzoe5Wzx06Kgjyxg@mail.gmail.com>
	<CA+7wUswJHYcCWQkb2-F7EtaRb13QkgqLrBvG7J1mEEwtp1DgNg@mail.gmail.com>
Message-ID: <CAFP7zVvZph6+bcbojffpVChzBh7izE-T4UD-KHfioge1qO9=mg@mail.gmail.com>

Mathieu,

thank you very much. This is exactly what I was still missing.

-Holger

On Thu, Sep 8, 2011 at 12:00, Mathieu Malaterre
<mathieu.malaterre at gmail.com> wrote:
> On Thu, Sep 8, 2011 at 11:56 AM, Holger Brandsmeier <brandsmeier at gmx.de> wrote:
> [...]
>> CMAKE_SHARED_LINKER_FLAGS. About the RTLD_GLOBAL flag for dlopen, I
>> don't know how python handles this.
>>
>> Do you have any ideas how to solve this? Or is this simply not
>> possible with python?
>
> Here is what I use:
>
> if os.name == 'posix':
> ?orig_dlopen_flags = sys.getdlopenflags()
> ?try:
> ? ?import dl
> ?except ImportError:
> ? ?try:
> ? ? ?import DLFCN as dl
> ? ?except ImportError:
> ? ? ?dl = None
> ?if dl:
> ? ?sys.setdlopenflags(dl.RTLD_NOW|dl.RTLD_GLOBAL)
> ?# now load the module !
> ?from gdcmswig import *
> ?sys.setdlopenflags(orig_dlopen_flags)
> ?del dl
> ?del orig_dlopen_flags
>
>
> ref:
> http://gdcm.git.sourceforge.net/git/gitweb.cgi?p=gdcm/gdcm;a=blob;f=Wrapping/Python/gdcm.py;h=fd7ee0da80297d608c2939b8661ce0e81c33314a;hb=HEAD
>
> --
> Mathieu
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From amundson at fnal.gov  Thu Sep  8 19:01:01 2011
From: amundson at fnal.gov (James Amundson)
Date: Thu, 08 Sep 2011 12:01:01 -0500
Subject: [C++-sig] How can I pass derived Python instances to C++?
Message-ID: <4E68F4CD.1030402@fnal.gov>

I would like to be able to create a Python class derived from a C++ 
class, then pass a instance of the Python class back to C++. It isn't 
clear to me if there is an easy way to do this.

Here is a simple example of what I wish worked, but doesn't:

C++ side:

#include <boost/python.hpp>
#include <iostream>
class Foo
{
public:
     Foo(){};
     virtual void
     doit(){std::cout << "doing it Foo style\n";};
};
void
foodoer(Foo & foo)
{
     foo.doit();
}
using namespace boost::python;
BOOST_PYTHON_MODULE(foo)
{
     class_<Foo >("Foo", init<>())
             .def("doit", &Foo::doit);
     def("foodoer", foodoer);
}

Python side:

import foo
class Bar(foo.Foo):
     def __init__(self):
         pass
     def doit(self):
         print "doing it bar style"

f = foo.Foo()
# next line works
foo.foodoer(f)

b = Bar()
# next line does not work
foo.foodoer(b)


Does anyone have any advice for me?

Thanks,
Jim Amundson


From stefan at seefeld.name  Thu Sep  8 19:17:50 2011
From: stefan at seefeld.name (Stefan Seefeld)
Date: Thu, 08 Sep 2011 13:17:50 -0400
Subject: [C++-sig] How can I pass derived Python instances to C++?
In-Reply-To: <4E68F4CD.1030402@fnal.gov>
References: <4E68F4CD.1030402@fnal.gov>
Message-ID: <4E68F8BE.9070006@seefeld.name>

On 2011-09-08 13:01, James Amundson wrote:
> I would like to be able to create a Python class derived from a C++
> class, then pass a instance of the Python class back to C++. It isn't
> clear to me if there is an easy way to do this.
>
>

> import foo
> class Bar(foo.Foo):
>     def __init__(self):
>         pass

You need to call the base class constructor explicitly in Python. That
should do the trick.

    Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From amundson at fnal.gov  Thu Sep  8 23:11:30 2011
From: amundson at fnal.gov (James Amundson)
Date: Thu, 08 Sep 2011 16:11:30 -0500
Subject: [C++-sig] How can I pass derived Python instances to C++?
In-Reply-To: <4E68F8BE.9070006@seefeld.name>
References: <4E68F4CD.1030402@fnal.gov> <4E68F8BE.9070006@seefeld.name>
Message-ID: <4E692F82.5040505@fnal.gov>

On 09/08/2011 12:17 PM, Stefan Seefeld wrote:
> On 2011-09-08 13:01, James Amundson wrote:
>> I would like to be able to create a Python class derived from a C++
>> class, then pass a instance of the Python class back to C++. It isn't
>> clear to me if there is an easy way to do this.
>>
>>
>> import foo
>> class Bar(foo.Foo):
>>      def __init__(self):
>>          pass
> You need to call the base class constructor explicitly in Python. That
> should do the trick.
Yes. Of course. Thanks! That should have been obvious to me.

In case anyone else is reading, I'll answer my own next question and say 
that Stefan's advice is necessary but not sufficient. In order to get my 
example to work, I had to read up on overridable virtual functions in 
Boost Python, e.g. here:
     http://wiki.python.org/moin/boost.python/OverridableVirtualFunctions

Here is my working example:

C++:

#include <boost/python.hpp>
#include <iostream>
class Foo
{
public:
     Foo()
     {
     }
     ;
     virtual void
     doit()
     {
         std::cout << "doing it Foo style\n";
     }
     ;
};
void
foodoer(Foo & foo)
{
     foo.doit();
}

using namespace boost::python;

struct Foo_callback : Foo
{
     Foo_callback(PyObject *p) :
         Foo(), self(p)
     {
     }
     Foo_callback(PyObject *p, const Foo& foo) :
         Foo(foo), self(p)
     {
     }
     void
     doit()
     {
         return call_method<void > (self, "doit");
     }
     static void
     default_doit(Foo& self_)
     {
         self_.Foo::doit();
     }
private:
     PyObject* self;
};

BOOST_PYTHON_MODULE(foo)
{
     class_<Foo, Foo_callback >("Foo", init<>())
     .def("doit", &Foo_callback::default_doit);
     def("foodoer", foodoer);
}

Python:

import foo
class Bar(foo.Foo):
     def __init__(self):
         foo.Foo.__init__(self)
     def doit(self):
         print "doing it Bar style"

f = foo.Foo()
foo.foodoer(f)

b = Bar()
foo.foodoer(b)


--Jim

From istvancsanady at gmail.com  Tue Sep 13 23:26:44 2011
From: istvancsanady at gmail.com (=?UTF-8?B?SXN0dsOhbiBDc2Fuw6FkeQ==?=)
Date: Tue, 13 Sep 2011 23:26:44 +0200
Subject: [C++-sig] scriptable widget system
Message-ID: <CAPX5cKxuWZqNoW6fLg2mJfEVO4wV62siPiPasZB_5vCEeOBoJQ@mail.gmail.com>

Hi All!

I am developing a custom widget system. The widgets are written in C++,
since the rendering requires very high performance. The widgets are loaded
from xml files. A widget can contain several different UI elements, for
example a slider, a textbox, a label, buttons, etc... Until now I used a
simple but quite powerful solution to synchronize the UI values and the
python object attributes. The concept is the following: C++ UI elements can
have properties (see the code below). Properties can be bounded to a python
object's attribute. If a property changes, it sets the python attribute, and
after the setter ran, it gets the actual python value, and sets the
property's value to it. (For better understanding: when a property changes
because of some user input it rather suggests a value then set it. This is
useful when for example you want a slider to be able to set only to odd
values, or want to limit the availible values of a numeric text box. So this
way you have full control of the "control UI elements" values.) From the
other side: when you set an attribute of the python object the object checks
if it is bounded to any C++ property, and if it is, then it sets the
property to the value. Here are some code snippets:

This is the C++ property:

template <typename T> class Property : public PythonPropertyBinder {



    public:



        void bindToAttribute(PythonComponent* component,std::string name) {

            //Add a PyCFunction to the bindings of the python object.

        }



        void unbind() {

            //Remove the bindings.

        }



        T& operator= (const Property<T>& p) {

            return (*this = p.value);

        }



        T & operator = (const T &i) {

            if(!updating) {

                updating = true;

                if(component) {

                    component->setAttribute(i, pythonAttributeName.c_str());

                    component->getAttribute(value,pythonAttributeName.c_str
());

                    PyErr_Print();

                }

                else {

                    value = i;

                }

                updating = false;

                notifyObservers();

                return value;

            }else {

                return value;

            }

        }



        operator T const & () const {

            return value;

        }





        void addObserver(PythonPropertyBinderObserver* observer) {

            observers.insert(observer);

        }



        void removeObserver(PythonPropertyBinderObserver* observer) {

            observers.erase(observer);

        }



        Property(const T& v):Property() {

            value = v;


        }



        Property():updating(false),component(nullptr) {

            //set up stuff...


        }



        ~Property() {

            unbind();

        }



    private:


        T value;

        std::string pythonAttributeName;



        //...not interesting

    };





And for example a slider looks like the following:


class Slider : public View, public PythonPropertyBinderObserver {



    public:

        //...


        Property<double> value;

        Property<double> low;

        Property<double> high;

        //...

};





The python class:



class Component(object):


    def bind_to_attribute(self,attribute_name, cpp_property_setter):

        if not hasattr(self,attribute_name):

            return

        if self.binder_functions.has_key(attribute_name):

            self.binder_functions[attribute_name].add(cpp_property_setter)

        else:

            s = set()

            s.add(cpp_property_setter)

            self.binder_functions[attribute_name] = s



    def unbind_from_attribute(self,attribute_name, cpp_property_setter):

        if self.binder_functions.has_key(attribute_name):

            self.binder_functions[attribute_name].remove(cpp_property_setter
)


    def __setattr__(self,attributename,value):

        super(Component,self).__setattr__(attributename,value)

        self.sync_attribute(attributename)


    def sync_attribute(self,attributename):

        if(hasattr(self,'binder_functions')
andself.binder_functions.has_key(attributename)):

            s = self.binder_functions[attributename]

            for cpp_property_setter in s:

                cpp_property_setter()


    '''

    And several other methods...

    '''


So this way I can simply bind a UI element's property to a python attribute,
and it is guaranteed that they will be the same (until a python property is
not accessed by its setter...)

For example:





class Slider(Component):



    def get_value(self):

        return self.__value



    def set_value(self,value):

        if((value<=self.high and value>=self.low) or (value>=self.high
andvalue<=self.low)):

            if self.integers_only:

                self.__value = round(value)

            else:

                self.__value = value

            self.run()



    def get_high(self):

        return self.__high



    def set_high(self,high):

        self.__high = high



    def get_low(self):

        return self.__low



    def set_low(self,low):

        self.low = low





    value = property(get_value,set_value)

    low = property(get_low,set_low)

    high = property(get_high,set_high)


And then all I have to do from the C++ UI:


    Slider* slider = new Slider(Point(20, 30.), 195.);



    slider->value.bindToAttribute(controller->getModel(),"value");

    slider->low.bindToAttribute(controller->getModel(),"low");

    slider->high.bindToAttribute(controller->getModel(),"high");




So this way:

- the python objects work like models

- the values are synced

- the script writer is not able access to any other properties of the UI
elements

- however one have total control of the values

- all the widgets can work without binded to any UI elements

- if I want to serialize the objects (and I want to), the PyCFunctions and
other - runtime added - objects can be easily excluded

- properties can be easily bounded and unbouded

- I did not have to create any new types, such as MySyncedFloat,
MySyncedString etc.

- the script writer doesn't even have to know about that a value will be
bounded to a UI element

- the UI elements are not strongly connected to any python code

- more UI elements can be bounded to a single python attribute for example I
can bind an attribute to a slider's max value and to a numeric text box at
the same time

- this approach can be easily used with a graphical interface builder: one
just have to drag and drop some UI elements to a widget, and tell the
interface builder that a property of an element will be bounded to a python
attribute. Then it can be exported to an XML and loaded runtime and bind the
properties runtime. Cool.



 This approach worked very well. But... Few days ago our UI/UX designer
created some new UI elements which I call "batched controllers". A batched
controller is an array of same controllers for example sliders. The number
of the sliders can be changed runtime for example when I click on a button a
new slider is added. And this type of view can not be used with the previous
approach, since a value is determined by two values: the ordinal of the
slider and the value itself.


So my question is: does anybody have any idea how to create something that
works like I previously specified for these "batched controllers"
(==properties with indices)


Istv?n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110913/550c57b9/attachment-0001.html>

From lists_ravi at lavabit.com  Sat Sep 17 17:15:22 2011
From: lists_ravi at lavabit.com (Ravi)
Date: Sat, 17 Sep 2011 08:15:22 -0700
Subject: [C++-sig] [Boost.Python v3] Features and Scope
In-Reply-To: <4E582E9F.6030808@gmail.com>
References: <4E56B7A6.2030008@gmail.com> <m2mxewkk8s.fsf@boostpro.com>
	<4E582E9F.6030808@gmail.com>
Message-ID: <201109170815.22287.lists_ravi@lavabit.com>

On Friday 26 August 2011 16:39:11 Jim Bosch wrote:
> > Well, speaking for myself, mostly time.  I'd be inclined to do a rewrite
> > along the lines of the langbinding ideas if I had time.
> 
> I had only been vaguely aware of langbinding until I followed up on your 
> last email.  It's a very nice separation, though after bad experiences 
> using SWIG I am a little wary about trying to build a one-size-fits-all 
> front-end for different languages.

Is code for boost.langbinding available anywhere? Or is it currently only a 
design description?

Regards,
Ravi


From dave at boostpro.com  Mon Sep 19 07:20:04 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Mon, 19 Sep 2011 01:20:04 -0400
Subject: [C++-sig] [Boost.Python v3] Features and Scope
References: <4E56B7A6.2030008@gmail.com> <m2mxewkk8s.fsf@boostpro.com>
	<4E582E9F.6030808@gmail.com>
	<201109170815.22287.lists_ravi@lavabit.com>
Message-ID: <m2litlp0qj.fsf@pluto.luannocracy.com>


on Sat Sep 17 2011, Ravi <lists_ravi-AT-lavabit.com> wrote:

> On Friday 26 August 2011 16:39:11 Jim Bosch wrote:
>> > Well, speaking for myself, mostly time.  I'd be inclined to do a rewrite
>
>> > along the lines of the langbinding ideas if I had time.
>> 
>> I had only been vaguely aware of langbinding until I followed up on your 
>> last email.  It's a very nice separation, though after bad experiences 
>> using SWIG I am a little wary about trying to build a one-size-fits-all 
>> front-end for different languages.
>
> Is code for boost.langbinding available anywhere? Or is it currently only a 
> design description?

Yes, there is some code, and there are tests, in
http://svn.boost.org/svn/boost/sandbox/langbinding/

HTH,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From talljimbo at gmail.com  Mon Sep 19 22:41:42 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Mon, 19 Sep 2011 16:41:42 -0400
Subject: [C++-sig] [Boost.Python v3] Planning and Logistics
In-Reply-To: <4E582E99.60001@gmail.com>
References: <4E56B7A6.2030008@gmail.com> <m2mxewkk8s.fsf@boostpro.com>
	<4E582E99.60001@gmail.com>
Message-ID: <4E77A906.7050004@gmail.com>

To summarize how I'm approaching this now:

- I think the cmake/github Boost repositories look like the right place 
to put this.

- I'm going to spend some time digesting the langbinding and luabind 
code, and try to put together a skeleton package; when it gets mature 
enough for others to find it useful to look at I'll upload it to github.

- In the meantime, I'll be re-starting some discussions on the list to 
flesh out some aspects that I don't have a clear vision for yet.

Jim

From talljimbo at gmail.com  Mon Sep 19 23:03:43 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Mon, 19 Sep 2011 17:03:43 -0400
Subject: [C++-sig] [Boost.Python v3] Conversions and Registries
Message-ID: <4E77AE2F.3070702@gmail.com>

I'd like to restart the discussion on how conversions will work in 
Boost.Python v3.  Here's the starting point:

I'd like to see support for static, template-based conversions.  These 
would be defined by [partial-]specializing a traits class, and I tend to 
think they should only be invoked after attempting all registry-based 
conversions.  Users would have to include the same headers in groups of 
interdependent modules to avoid specializing the same traits class 
multiple times in different ways; I can't think of a way to protect them 
from this, but template-based specializations are a sufficiently 
advanced featured that I'm comfortable leaving it up to users to avoid 
this problem.

We've had some discussion of allowing different modules to have 
different registries (in addition to a global registry shared across 
modules).  Leaving aside implementation questions, I have a little 
survey for interested parties:

1) Under what circumstances would you want a conversion that is 
completely limited to a specific module (or a group of modules that 
explicitly declare it)?

2) Under what circumstances would you want a conversion to look in a 
module-specific registry, and then fall back to the global registry?

3) Considering that we will have a "best-match" overloading system, what 
should take precedence, an inexact match in a module-specific registry, 
or an exact match in a global registry?  (Clearly this is a moot point 
for to-Python conversion).

Finally, can anyone give me a reason why having a global registry can 
lead to a violation of the "One Definition Rule"?  This was alluded to 
many times in the earlier discussion, and there's no doubt that a global 
registry may lead to unexpected (from a given module's perspective) 
behavior - but I do not understand the implication that the global 
registry can result in formally undefined behavior by violating the ODR.

Thanks!

Jim

From brandsmeier at gmx.de  Mon Sep 19 23:32:37 2011
From: brandsmeier at gmx.de (Holger Brandsmeier)
Date: Mon, 19 Sep 2011 23:32:37 +0200
Subject: [C++-sig] [Boost.Python v3] Conversions and Registries
In-Reply-To: <4E77AE2F.3070702@gmail.com>
References: <4E77AE2F.3070702@gmail.com>
Message-ID: <CAFP7zVv5LBVEBLqFoyboN5_+Q1FjePUdbZCZbnBgUARa_tE9tg@mail.gmail.com>

Jim,

My answer for 1), 2) for module specific behavior would be that I
would probably not use module specific behavior. In my current project
I separate the code into multiple BOOST_PYTHON_MODULE. So I would want
to use project specific behavior and not behavior specific to the
BOOST_PYTHON_MODULE. I could reorganize the code, but I kind of like
the idea of separating the big python module into several sub-modules.

As for project specific behavior, I would actually want 2) always, and
I would want 1) optionally. So I would like to enable or disable 1)
either in the build tool or with some global python variables.

As for 3) I would prefer the exact match from the global registry. I
assume that this case would appear if a user overloads a function from
python.

-Holger

On Mon, Sep 19, 2011 at 23:03, Jim Bosch <talljimbo at gmail.com> wrote:
> I'd like to restart the discussion on how conversions will work in
> Boost.Python v3. ?Here's the starting point:
>
> I'd like to see support for static, template-based conversions. ?These would
> be defined by [partial-]specializing a traits class, and I tend to think
> they should only be invoked after attempting all registry-based conversions.
> ?Users would have to include the same headers in groups of interdependent
> modules to avoid specializing the same traits class multiple times in
> different ways; I can't think of a way to protect them from this, but
> template-based specializations are a sufficiently advanced featured that I'm
> comfortable leaving it up to users to avoid this problem.
>
> We've had some discussion of allowing different modules to have different
> registries (in addition to a global registry shared across modules).
> ?Leaving aside implementation questions, I have a little survey for
> interested parties:
>
> 1) Under what circumstances would you want a conversion that is completely
> limited to a specific module (or a group of modules that explicitly declare
> it)?
>
> 2) Under what circumstances would you want a conversion to look in a
> module-specific registry, and then fall back to the global registry?
>
> 3) Considering that we will have a "best-match" overloading system, what
> should take precedence, an inexact match in a module-specific registry, or
> an exact match in a global registry? ?(Clearly this is a moot point for
> to-Python conversion).
>
> Finally, can anyone give me a reason why having a global registry can lead
> to a violation of the "One Definition Rule"? ?This was alluded to many times
> in the earlier discussion, and there's no doubt that a global registry may
> lead to unexpected (from a given module's perspective) behavior - but I do
> not understand the implication that the global registry can result in
> formally undefined behavior by violating the ODR.
>
> Thanks!
>
> Jim
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From s_sourceforge at nedprod.com  Tue Sep 20 17:06:57 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Tue, 20 Sep 2011 16:06:57 +0100
Subject: [C++-sig] [Boost.Python v3] Conversions and Registries
In-Reply-To: <4E77AE2F.3070702@gmail.com>
References: <4E77AE2F.3070702@gmail.com>
Message-ID: <4E78AC11.5043.B0CBE312@s_sourceforge.nedprod.com>

On 19 Sep 2011 at 17:03, Jim Bosch wrote:

> I'd like to see support for static, template-based conversions.  These 
> would be defined by [partial-]specializing a traits class, and I tend to 
> think they should only be invoked after attempting all registry-based 
> conversions. 

Surely not! You'd want to let template specialisaton be the first 
point of call so the compiler can compile in obvious conversions, 
*then* and only then do you go to a runtime registry.

This also lets one override the runtime registry when needed in the 
local compiland. I'm not against having another set of template 
specialisations do something should the first set of specialisations 
fail, and/or the runtime registry lookup fails.

> Users would have to include the same headers in groups of 
> interdependent modules to avoid specializing the same traits class 
> multiple times in different ways; I can't think of a way to protect them 
> from this, but template-based specializations are a sufficiently 
> advanced featured that I'm comfortable leaving it up to users to avoid 
> this problem.

Just make sure what you do works with precompiled headers :)

P.S.: This is trickier than it sounds.

> We've had some discussion of allowing different modules to have 
> different registries (in addition to a global registry shared across 
> modules).  Leaving aside implementation questions, I have a little 
> survey for interested parties:
> 
> 1) Under what circumstances would you want a conversion that is 
> completely limited to a specific module (or a group of modules that 
> explicitly declare it)?

Defaults to most recent in calling thread stack, but overridable 
using a TLS override to allow impersonation.

The same mechanism usefully also takes care of multiple python 
interpreters too.

> 2) Under what circumstances would you want a conversion to look in a 
> module-specific registry, and then fall back to the global registry?

As above. That implies that there is no global registry, just the 
default registry which all module registries inherit.

> 3) Considering that we will have a "best-match" overloading system, what 
> should take precedence, an inexact match in a module-specific registry, 
> or an exact match in a global registry?  (Clearly this is a moot point 
> for to-Python conversion).

The way I've always done this is to have the template metaprogramming 
set a series of type comparison functions which return scores. This 
pushes most of the scoring and weighting into the compiler and the 
compiler will elide any calls into the dynamic registry where the 
scoring makes that sensible. Makes compile times rather longer though 
:)

The dynamic and compile-time registries can be merged easily enough, 
so all the runtime registry is is a set of comparison functions 
normally elided by the compiler in other modules. In other words, 
mark the inline functions as visible outside the current DLL 
(dllexport/visibility(default)) so the compiler will assemble 
complete versions for external usage.

> Finally, can anyone give me a reason why having a global registry can 
> lead to a violation of the "One Definition Rule"?  This was alluded to 
> many times in the earlier discussion, and there's no doubt that a global 
> registry may lead to unexpected (from a given module's perspective) 
> behavior - but I do not understand the implication that the global 
> registry can result in formally undefined behavior by violating the ODR.

ODR only matters in practice for anything visible outside the current 
compiland. If compiling with GCC -fvisibility=hidden, or on any MSVC 
by default, you can define class foo to be anything you like so long 
as nothing outside the current compiland can see class foo.

ODR is real important though across DLLs. If a DLL X says that class 
foo is one thing and DLL Y says it's something different, expect 
things to go very badly wrong. Hence I simply wouldn't have a global 
registry. It's bad design. You *have* to have per module registries 
and *only* per module registries.

Imagine the following. Program A loads DLL B and DLL C. DLL B is 
dependent on DLL D which uses BPL. DLL C is dependent on DLL E which 
uses BPL.

DLL D tells BPL that class foo is implicitly convertible with an 
integer.

DLL E tells BPL that class foo is actually a thin wrapper for 
std::string.

Right now with present BPL, we have to load two copies of BPL, one 
for DLL D and one for DLL E. They maintain separate type registries, 
so all is good.

But what if DLL B returns a python function to Program A, which then 
installs it as a callback with DLL C?

In the normal case, BPL code in DLL E will call into BPL code DLL D 
and all is well.

But what if the function in DLL D throws an exception?

This gets converted into a C++ exception by throwing 
boost::error_already_set.

Now the C++ runtime must figure where to send the exception. But what 
is the C++ runtime supposed to do with such an exception type? It 
isn't allowed to see the copy of BPL living in DLL E, so it will fire 
the exception type into DLL D where it doesn't belong. At this point, 
the program will almost certainly segfault.

Whatever you do with BPL in the future, it MUST support being a 
dependency of multiple DLLs simultaneously. It MUST know who is 
calling what and when, and know how to unwind everything at any 
particular stage. This implies that it must be 100% compatible with 
dlopen(RTLD_GLOBAL).

As I mentioned earlier, this is a very semantically similar problem 
to supporting multiple python interpreters anyway with each calling 
into one another. You can kill two birds with the one stone as a 
result.

HTH,
Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From talljimbo at gmail.com  Tue Sep 20 18:38:14 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 20 Sep 2011 12:38:14 -0400
Subject: [C++-sig] [Boost.Python v3] Conversions and Registries
In-Reply-To: <4E78AC11.5043.B0CBE312@s_sourceforge.nedprod.com>
References: <4E77AE2F.3070702@gmail.com>
	<4E78AC11.5043.B0CBE312@s_sourceforge.nedprod.com>
Message-ID: <4E78C176.7090305@gmail.com>

On 09/20/2011 11:06 AM, Niall Douglas wrote:
> On 19 Sep 2011 at 17:03, Jim Bosch wrote:
>
>> I'd like to see support for static, template-based conversions.  These
>> would be defined by [partial-]specializing a traits class, and I tend to
>> think they should only be invoked after attempting all registry-based
>> conversions.
>
> Surely not! You'd want to let template specialisaton be the first
> point of call so the compiler can compile in obvious conversions,
> *then* and only then do you go to a runtime registry.
>
> This also lets one override the runtime registry when needed in the
> local compiland. I'm not against having another set of template
> specialisations do something should the first set of specialisations
> fail, and/or the runtime registry lookup fails.
>

I'd also considered having a different set of template conversions that 
are checked first for performance reasons, but I'd actually viewed the 
override preference argument from the opposite direction - once a 
template converter traits class has been fully specialized, you can't 
specialize it again differently in another module (well, maybe symbol 
visibility labels can get you out of that bind in practice).  So it 
seemed a registry-based override would be the only way to override a 
template-based conversion, and hence the registry-based conversions 
would have to go first.

But overall I think your proposal to just try the templates first is 
cleaner, because having multiple specializations of the same traits 
class in different modules would be a problem either way; allowing users 
to override the compile-time conversions with registry-based conversions 
is at best a poor workaround.

>> Users would have to include the same headers in groups of
>> interdependent modules to avoid specializing the same traits class
>> multiple times in different ways; I can't think of a way to protect them
>> from this, but template-based specializations are a sufficiently
>> advanced featured that I'm comfortable leaving it up to users to avoid
>> this problem.
>
> Just make sure what you do works with precompiled headers :)
>
> P.S.: This is trickier than it sounds.
>

Yuck.  Precompiled headers are something I've never dealt with before, 
but I suppose I had better learn.

>> We've had some discussion of allowing different modules to have
>> different registries (in addition to a global registry shared across
>> modules).  Leaving aside implementation questions, I have a little
>> survey for interested parties:
>>
>> 1) Under what circumstances would you want a conversion that is
>> completely limited to a specific module (or a group of modules that
>> explicitly declare it)?
>
> Defaults to most recent in calling thread stack, but overridable
> using a TLS override to allow impersonation.
>
> The same mechanism usefully also takes care of multiple python
> interpreters too.
>

I have to admit I'm only barely following you here - threads are another 
thing I don't deal with often.  It sounds like you have a totally 
different option from the ones I was anticipating.  Could you explain in 
more detail how this would work?

>> 2) Under what circumstances would you want a conversion to look in a
>> module-specific registry, and then fall back to the global registry?
>
> As above. That implies that there is no global registry, just the
> default registry which all module registries inherit.

(still a little confused about what you mean)

>> 3) Considering that we will have a "best-match" overloading system, what
>> should take precedence, an inexact match in a module-specific registry,
>> or an exact match in a global registry?  (Clearly this is a moot point
>> for to-Python conversion).
>
> The way I've always done this is to have the template metaprogramming
> set a series of type comparison functions which return scores. This
> pushes most of the scoring and weighting into the compiler and the
> compiler will elide any calls into the dynamic registry where the
> scoring makes that sensible. Makes compile times rather longer though
> :)
>
> The dynamic and compile-time registries can be merged easily enough,
> so all the runtime registry is is a set of comparison functions
> normally elided by the compiler in other modules. In other words,
> mark the inline functions as visible outside the current DLL
> (dllexport/visibility(default)) so the compiler will assemble
> complete versions for external usage.
>

An interesting idea - avoid trying all possible conversions a runtime 
seems a very worthy goal, though I could also see this inflating the 
size of the modules.  Can you point me at anything existing for an example?

>> Finally, can anyone give me a reason why having a global registry can
>> lead to a violation of the "One Definition Rule"?  This was alluded to
>> many times in the earlier discussion, and there's no doubt that a global
>> registry may lead to unexpected (from a given module's perspective)
>> behavior - but I do not understand the implication that the global
>> registry can result in formally undefined behavior by violating the ODR.
>
> ODR only matters in practice for anything visible outside the current
> compiland. If compiling with GCC -fvisibility=hidden, or on any MSVC
> by default, you can define class foo to be anything you like so long
> as nothing outside the current compiland can see class foo.
>
> ODR is real important though across DLLs. If a DLL X says that class
> foo is one thing and DLL Y says it's something different, expect
> things to go very badly wrong. Hence I simply wouldn't have a global
> registry. It's bad design. You *have* to have per module registries
> and *only* per module registries.
> Imagine the following. Program A loads DLL B and DLL C. DLL B is
> dependent on DLL D which uses BPL. DLL C is dependent on DLL E which
> uses BPL.
>
> DLL D tells BPL that class foo is implicitly convertible with an
> integer.
>
> DLL E tells BPL that class foo is actually a thin wrapper for
> std::string.
>
> Right now with present BPL, we have to load two copies of BPL, one
> for DLL D and one for DLL E. They maintain separate type registries,
> so all is good.
>
> But what if DLL B returns a python function to Program A, which then
> installs it as a callback with DLL C?
>
> In the normal case, BPL code in DLL E will call into BPL code DLL D
> and all is well.
>
> But what if the function in DLL D throws an exception?
>
> This gets converted into a C++ exception by throwing
> boost::error_already_set.
>
> Now the C++ runtime must figure where to send the exception. But what
> is the C++ runtime supposed to do with such an exception type? It
> isn't allowed to see the copy of BPL living in DLL E, so it will fire
> the exception type into DLL D where it doesn't belong. At this point,
> the program will almost certainly segfault.
>
> Whatever you do with BPL in the future, it MUST support being a
> dependency of multiple DLLs simultaneously. It MUST know who is
> calling what and when, and know how to unwind everything at any
> particular stage. This implies that it must be 100% compatible with
> dlopen(RTLD_GLOBAL).
>
> As I mentioned earlier, this is a very semantically similar problem
> to supporting multiple python interpreters anyway with each calling
> into one another. You can kill two birds with the one stone as a
> result.

If I understand your argument, it's not the global registry that causes 
ODR violations - it's the fact that you're trying to mimic having local 
registries by forcing distinct BPLs for each module, and that makes BPL 
symbols ambiguous.  If you had a pair of modules that were happy using 
each other's converters, they would do the standard thing and share one 
BPL and one registry and you wouldn't have any ODR problems.

In other words, it's not the fact that DLL D and DLL E register 
different conversions for class foo that causes the ODR problems; that 
just makes modules interact unfortunately (but in a deterministic and 
debuggable way).  It's the workaround (loading multiple BPLs) that 
causes the actual ODR problems.

So it sounds we agree that we should only ever have one BPL loaded.  We 
just need to implement the registry so it can know which module DLL 
instance a particular registry lookup is coming from, whether that's 
using special module-instance IDs or compiling the registries into the 
module DLLs or something else.

Is that right?

Thanks!

Jim

From s_sourceforge at nedprod.com  Tue Sep 20 20:36:57 2011
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Tue, 20 Sep 2011 19:36:57 +0100
Subject: [C++-sig] [Boost.Python v3] Conversions and Registries
In-Reply-To: <4E78C176.7090305@gmail.com>
References: <4E77AE2F.3070702@gmail.com>,
	<4E78AC11.5043.B0CBE312@s_sourceforge.nedprod.com>,
	<4E78C176.7090305@gmail.com>
Message-ID: <4E78DD49.21374.B18C2499@s_sourceforge.nedprod.com>

On 20 Sep 2011 at 12:38, Jim Bosch wrote:

> I'd also considered having a different set of template conversions that 
> are checked first for performance reasons, but I'd actually viewed the 
> override preference argument from the opposite direction - once a 
> template converter traits class has been fully specialized, you can't 
> specialize it again differently in another module (well, maybe symbol 
> visibility labels can get you out of that bind in practice).  So it 
> seemed a registry-based override would be the only way to override a 
> template-based conversion, and hence the registry-based conversions 
> would have to go first.

Ah, sorry, I didn't explain myself well at all. I've been doing a lot 
of work surrounding ISO C and C++ standards recently, so my head is 
kinda trapped in future C and C++. When I was speaking of ODR, I was 
kinda assuming that we have C++ modules available for newer compilers 
in the post C++-1x TR (see http://www.open-
std.org/jtc1/sc22/wg21/docs/papers/2007/n2316.pdf) and we can emulate 
much of module support using -fvisibility=hidden on GCC. On MSVC, of 
course, you get module proto-support for free anyway due to how their 
DLLs work.

You're absolutely correct that right now, outside of the Windows 
platform, ODR is a process wide problem in most compilers on their 
default settings. That's a PITA, so everyone is agreed that we ought 
to do something about it. The big problem is how far we ought to go, 
hence N2316 not making it into C++-1x and being pushed into TR.

What I can say is that that TR will very likely be highly compatible 
with the Windows DLL system (and its GCC visibility near-equivalent) 
due to backwards compatibility. I would suggest that you code as if 
both are true as a reasonable proxy for future C++ module support. 
Then you're covered ten years down the line from now.

> But overall I think your proposal to just try the templates first is 
> cleaner, because having multiple specializations of the same traits 
> class in different modules would be a problem either way; allowing users 
> to override the compile-time conversions with registry-based conversions 
> is at best a poor workaround.

I know this is a little off-topic, but Boost could really do with a 
generic runtime type registry implementation. There are lots of use 
cases outside BPL and if we had one, highly extensible, properly 
written system it could be applied to lots of use cases.

For example, Java-style automagical metaprogrammed C++ type 
reflection into SQL is perfectly possible. At the time I wrote it, it 
was the only example of it anywhere I could find (maybe things have 
since changed). It makes talking to databases super-easy at the cost 
of making the compiler work very hard.

There are lots more use cases too e.g. talking with .NET, or 
Objective C.

> > Just make sure what you do works with precompiled headers :)
> >
> > P.S.: This is trickier than it sounds.
> 
> Yuck.  Precompiled headers are something I've never dealt with before, 
> but I suppose I had better learn.

Getting them working can make the difference between a several hour 
recompile and ten minutes. They're painful though due to compiler 
bugs.

> > The same mechanism usefully also takes care of multiple python
> > interpreters too.
> 
> I have to admit I'm only barely following you here - threads are another 
> thing I don't deal with often.  It sounds like you have a totally 
> different option from the ones I was anticipating.  Could you explain in 
> more detail how this would work?

Sure. You have the problem when working with Python of handling the 
GIL which is strongly related to what the "current" interpreter is. 
These are TLS items in python, so each thread has its own current 
setting.

Therefore, what one really ought to have in BPL is something like:

// normal C++ code
...
// I want to call python code in interpreter X
{
  boost::python::hold_interpreter interpreter_holder(X); // Replaces 
"current" interpreter with X
  boost::python::hold_GIL gil_holder(interpreter_holder); // Acquire 
the GIL for that interpreter
  call_some_BPL_or_python_function();
} // On scope exit gil_holder and interpreter_holder gets destroyed, 
thus releasing the GIL and resetting the "current" interpreter to 
whatever it was before
// Back to normal C++ code

This obviously refers to the embedded case, but it ought to be 
similar when BPL calls into C++: the "current" interpreter should be 
available per thread as a BPL object instance wrapping the python TLS 
config. Then a call into C++ can safely call into other interpreters.

What's useful here of course is that you can keep a per-thread list 
of interpreter nestings. This means you can see exactly which module 
entered which interpreter and in which order, and therefore what to 
search and what to unwind when necessary.

> An interesting idea - avoid trying all possible conversions a runtime 
> seems a very worthy goal, though I could also see this inflating the 
> size of the modules.  Can you point me at anything existing for an example?

The closest that I have publicly available is the SQL type reflection 
machinery in TnFOX. Have a look at the following:

https://github.com/ned14/tnfox/blob/master/include/TnFXSQLDB.h
https://github.com/ned14/tnfox/blob/master/include/TnFXSQLDB_ipc.h
https://github.com/ned14/tnfox/blob/master/include/TnFXSQLDB_sqlite3.h

Note that this is an entirely *static* type registry, so it exists 
exclusively in the compiler.

It does happily extend into a dynamic registry however. I can supply 
the source which extends the TnFOX static registry with a dynamic 
runtime, but I'd need you to agree to an NDA and a promise not to 
distribute them.

> If I understand your argument, it's not the global registry that causes 
> ODR violations - it's the fact that you're trying to mimic having local 
> registries by forcing distinct BPLs for each module, and that makes BPL 
> symbols ambiguous.  If you had a pair of modules that were happy using 
> each other's converters, they would do the standard thing and share one 
> BPL and one registry and you wouldn't have any ODR problems.

ODR is a C++ (and C) spec issue and has nothing to do with BPL per 
se. It's rather that because real world code routinely violates ODR 
that it becomes a problem for anything which operates a type 
registry.

BTW code can't help violating it. Libraries have absolutely no 
control over what they must coexist with in a given process.

> In other words, it's not the fact that DLL D and DLL E register 
> different conversions for class foo that causes the ODR problems; that 
> just makes modules interact unfortunately (but in a deterministic and 
> debuggable way).  It's the workaround (loading multiple BPLs) that 
> causes the actual ODR problems.

RTLD_GLOBAL operates okay for most C++ programs because that's the 
default. Indeed, until very recently, GCC couldn't throw exceptions 
properly unless RTLD_GLOBAL was set.

Unfortunately, Python sets RTLD_LOCAL for the process because up 
until I patched GCC to add -fvisibility, there was no easy way to 
separate Python extension modules from one another. They routinely 
defined functions with identical symbols and therefore one got all 
sorts of unpleasant conflicts.

One therefore gets a big problem when using anything C++ with a type 
registry within Python. One typically has to resort to unpleasant 
hacking of dlopen settings.

> So it sounds we agree that we should only ever have one BPL loaded.  We 
> just need to implement the registry so it can know which module DLL 
> instance a particular registry lookup is coming from, whether that's 
> using special module-instance IDs or compiling the registries into the 
> module DLLs or something else.
> 
> Is that right?

Ah, but it gets worse! You can't guarantee that BPL won't be loaded 
multiply anyway. For example, one might have dependencies on two 
separate versions of BPL, or some sublibrary might link a copy of BPL 
in statically.

In fact, you can't even guarantee that there aren't multiple pythons 
running! One (nasty) way of implementing parallel python is to 
instantiate multiple pythons each with their own GIL and run them in 
separate threads. Of course, forking yourself is far saner.

In the end though, BPL is a *library*. You have absolutely no control 
over what you're combined with, but you can try your best for most 
reasonable scenarios.

I know this sounds tricky, but what you need is a design which copes 
with having one BPL loaded or many and/or one python loaded or many 
and/or one interpreter running or many. If you follow the system 
described above where each thread keeps a list of which BPL and 
python interpreter is "current", you now know which type registries 
to search in any given scenario.

You can see most of an existing implementation of what I described 
above at:

https://github.com/ned14/tnfox/blob/master/Python/FXPython.h
https://github.com/ned14/tnfox/blob/master/Python/FXPython.cxx


And oh, BTW, here is a very useful piece of C++ metaprogramming for 
BPL:

https://github.com/ned14/tnfox/blob/master/Python/FXCodeToPythonCode.h

This lets you handle a limitation in present BPL where you want to 
supply one of a list of python functions as a C callback function 
e.g. a comparison function for sorting. The metaprogramming generates 
a N member jump table and you supply a policy which thunks the C 
callback into Python. You can then install or deinstall python 
functions to the "slot" as it were and pass the appropriate C wrapper 
to the C callback function taking code.

In other words, the metaprogramming generates a unique C function 
address for each unique Python function (for the possibilities 
supplied). This is extremely useful.

Hope these help. If you have any questions, please do ask. I always 
felt it a shame I never had the time to port TnFOX extensions to BPL 
back into Boost, kinda wasted me writing it all as no one uses TnFOX 
:(

Niall

-- 
Technology & Consulting Services - ned Productions Limited.
http://www.nedproductions.biz/. VAT reg: IE 9708311Q. Company no: 
472909.




From k.hughes at queensu.ca  Wed Sep 21 15:31:20 2011
From: k.hughes at queensu.ca (Kevin Hughes)
Date: Wed, 21 Sep 2011 09:31:20 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv 2.3
In-Reply-To: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
Message-ID: <CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>

I am trying to create a python library from a class which uses opencv 2.3. I
want to be able to pass numpy array's into the class where they will be
converted into cv::Mat's processed then converted back to numpy array's and
returned.

Here is a simple test class I am working on to get this working before
wrapping my own class. Currently I am just trying to receive a numpy array
concert to a cv::Mat, process it and then write it to file. After this is
working I will work on returning the processed array to python.

Here is the simple class:

foo.h :

#include <opencv2/core/core.hpp>

 class Foo {

    public:

        Foo();

        ~Foo();

        cv::Mat image;

        void bar( cv::Mat in );
};

foo.cpp :

  #include "foo.h"

  Foo::Foo(){}

  Foo::~Foo(){}

  void Foo::bar( cv::Mat in) {

      image = in;

      cv::Canny( image, image, 50, 100 );

      cv::imwrite("image.png", image);

  }

And here is where I have attempted to wrap this class using boost::python (I
am using code from the opencv source for the the numpy to mat conversion)

wrap_foo.cpp

#include <boost/python.hpp>
#include <numpy/arrayobject.h>

#include <opencv2/core/core.hpp>

#include "foo.h"

using namespace cv;
namespace bp = boost::python;

//// Wrapper Functions
void bar(Foo& f, bp::object np);

//// Converter Functions

cv::Mat convertNumpy2Mat(bp::object np);

//// Wrapper Functions
void bar(Foo& f, bp::object np)
{

    Mat img = convertNumpy2Mat(np);

    f.bar(img);

    return;
}


//// Boost Python Class

BOOST_PYTHON_MODULE(lib)
{

    bp::class_<Foo>("Foo")

        .def("bar", bar)

        ;
}


//// Converters

cv::Mat convertNumpy2Mat(bp::object np)
{

   Mat m;

   numpy_to_mat(np.ptr(),m);

   return m;
}

The numpy_to_mat function is from the opencv source
(modules/python/src2/cv2.cpp). The full file has the function below what I
wrote above. This code compiles with bjam just fine but the when I import
into python it crashes. The error is this: libFoo.so: undefined symbol:
_ZN2cv3Mat10deallocateEv. I have tried a number of different things but I
can't get this to work.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110921/920e799c/attachment-0001.html>

From talljimbo at gmail.com  Wed Sep 21 15:43:57 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Wed, 21 Sep 2011 09:43:57 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv
 2.3
In-Reply-To: <CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
	<CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
Message-ID: <4E79EA1D.1010405@gmail.com>

On 09/21/2011 09:31 AM, Kevin Hughes wrote:
> I am trying to create a python library from a class which uses opencv
> 2.3. I want to be able to pass numpy array's into the class where they
> will be converted into cv::Mat's processed then converted back to numpy
> array's and returned.
>
> Here is a simple test class I am working on to get this working before
> wrapping my own class. Currently I am just trying to receive a numpy
> array concert to a cv::Mat, process it and then write it to file. After
> this is working I will work on returning the processed array to python.
>

Without digging too deeply or getting into possibly better solutions, 
try adding:

import_array();

inside your BOOST_PYTHON_MODULE block.  Or maybe there's some opencv 
initialization function you need to call that will do that for you.  The 
NumPy C-API (which I assume the numpy_to_mat function uses internally) 
needs to be initialized when your module is imported, or you'll get 
segfaults.  The above call to import_array() is how you'd do it if you 
were using the NumPy C-API headers directly.  In any case, you need to 
make sure it gets called somehow.

HTH

Jim Bosch

From k.hughes at queensu.ca  Wed Sep 21 16:25:52 2011
From: k.hughes at queensu.ca (Kevin Hughes)
Date: Wed, 21 Sep 2011 10:25:52 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv
	2.3
In-Reply-To: <4E79EA1D.1010405@gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
	<CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
	<4E79EA1D.1010405@gmail.com>
Message-ID: <CANY4+HX46dPS4RFcUN5PNzCduEcRrnT=EzXQjbGXfyG-r+JFvQ@mail.gmail.com>

Thanks for the reply, I tried adding import_array(); to my
BOOST_PYTHON_MODULE block but I still got an error when importing my library
into python, the same same error I mentioned above.

On Wed, Sep 21, 2011 at 9:43 AM, Jim Bosch <talljimbo at gmail.com> wrote:

> On 09/21/2011 09:31 AM, Kevin Hughes wrote:
>
>> I am trying to create a python library from a class which uses opencv
>> 2.3. I want to be able to pass numpy array's into the class where they
>> will be converted into cv::Mat's processed then converted back to numpy
>> array's and returned.
>>
>> Here is a simple test class I am working on to get this working before
>> wrapping my own class. Currently I am just trying to receive a numpy
>> array concert to a cv::Mat, process it and then write it to file. After
>> this is working I will work on returning the processed array to python.
>>
>>
> Without digging too deeply or getting into possibly better solutions, try
> adding:
>
> import_array();
>
> inside your BOOST_PYTHON_MODULE block.  Or maybe there's some opencv
> initialization function you need to call that will do that for you.  The
> NumPy C-API (which I assume the numpy_to_mat function uses internally) needs
> to be initialized when your module is imported, or you'll get segfaults.
>  The above call to import_array() is how you'd do it if you were using the
> NumPy C-API headers directly.  In any case, you need to make sure it gets
> called somehow.
>
> HTH
>
> Jim Bosch
> ______________________________**_________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110921/9bf7635c/attachment.html>

From talljimbo at gmail.com  Wed Sep 21 16:41:14 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Wed, 21 Sep 2011 10:41:14 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv
 2.3
In-Reply-To: <CANY4+HX46dPS4RFcUN5PNzCduEcRrnT=EzXQjbGXfyG-r+JFvQ@mail.gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
	<CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
	<4E79EA1D.1010405@gmail.com>
	<CANY4+HX46dPS4RFcUN5PNzCduEcRrnT=EzXQjbGXfyG-r+JFvQ@mail.gmail.com>
Message-ID: <4E79F78A.9080908@gmail.com>

On 09/21/2011 10:25 AM, Kevin Hughes wrote:
> Thanks for the reply, I tried adding import_array(); to my
> BOOST_PYTHON_MODULE block but I still got an error when importing my
> library into python, the same same error I mentioned above.

Does it actually crash Python, or just fail to import the module?  It 
looks like a link error, and that usually just causes the module to fail 
to import.  And that would suggest looking at your build system for 
whether you've linked properly against the opencv libs.

Jim


From k.hughes at queensu.ca  Wed Sep 21 17:12:37 2011
From: k.hughes at queensu.ca (Kevin Hughes)
Date: Wed, 21 Sep 2011 11:12:37 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv
	2.3
In-Reply-To: <4E79F78A.9080908@gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
	<CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
	<4E79EA1D.1010405@gmail.com>
	<CANY4+HX46dPS4RFcUN5PNzCduEcRrnT=EzXQjbGXfyG-r+JFvQ@mail.gmail.com>
	<4E79F78A.9080908@gmail.com>
Message-ID: <CANY4+HXWx1UNJCm3c_VFrtDKQi9_6suLoBHWbAwV7aPsRP6i9w@mail.gmail.com>

Sorry it just fails to import, what should my JamRoot file look like to
include my opencv libs? I am very new to boost python.

On Wed, Sep 21, 2011 at 10:41 AM, Jim Bosch <talljimbo at gmail.com> wrote:

> On 09/21/2011 10:25 AM, Kevin Hughes wrote:
>
>> Thanks for the reply, I tried adding import_array(); to my
>> BOOST_PYTHON_MODULE block but I still got an error when importing my
>> library into python, the same same error I mentioned above.
>>
>
> Does it actually crash Python, or just fail to import the module?  It looks
> like a link error, and that usually just causes the module to fail to
> import.  And that would suggest looking at your build system for whether
> you've linked properly against the opencv libs.
>
> Jim
>
> ______________________________**_________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110921/47830547/attachment.html>

From k.hughes at queensu.ca  Wed Sep 21 17:13:24 2011
From: k.hughes at queensu.ca (Kevin Hughes)
Date: Wed, 21 Sep 2011 11:13:24 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv
	2.3
In-Reply-To: <CANY4+HXWx1UNJCm3c_VFrtDKQi9_6suLoBHWbAwV7aPsRP6i9w@mail.gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
	<CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
	<4E79EA1D.1010405@gmail.com>
	<CANY4+HX46dPS4RFcUN5PNzCduEcRrnT=EzXQjbGXfyG-r+JFvQ@mail.gmail.com>
	<4E79F78A.9080908@gmail.com>
	<CANY4+HXWx1UNJCm3c_VFrtDKQi9_6suLoBHWbAwV7aPsRP6i9w@mail.gmail.com>
Message-ID: <CANY4+HUu4wvU3tEWEBj-mKanWXRbo+3Fh790UuPCa05OROBjAQ@mail.gmail.com>

Here is the JamRoot file I have been using:

using python ;

# Specify that the boost-python library exists under the name
# boost_python. That is, because the library was installed at the
# standard search path as /usr/lib/libboost_python.so, bjam will find
# it automatically. No need to specify the absolute path.
# lib libboost_python : : <name>boost_python-mt ;
lib libboost_python : : <name>boost_python ;
lib libboost_date_time : : <name>boost_date_time ;

# Set up the project-wide requirements that everything uses the
# boost_python library.
project
    : requirements <library>libboost_python
;

# Declare the extension modules.  You can specify multiple
# source files after the colon separated by spaces.
python-extension libFoo : wrap_foo.cpp ;



On Wed, Sep 21, 2011 at 11:12 AM, Kevin Hughes <k.hughes at queensu.ca> wrote:

> Sorry it just fails to import, what should my JamRoot file look like to
> include my opencv libs? I am very new to boost python.
>
>
> On Wed, Sep 21, 2011 at 10:41 AM, Jim Bosch <talljimbo at gmail.com> wrote:
>
>> On 09/21/2011 10:25 AM, Kevin Hughes wrote:
>>
>>> Thanks for the reply, I tried adding import_array(); to my
>>> BOOST_PYTHON_MODULE block but I still got an error when importing my
>>> library into python, the same same error I mentioned above.
>>>
>>
>> Does it actually crash Python, or just fail to import the module?  It
>> looks like a link error, and that usually just causes the module to fail to
>> import.  And that would suggest looking at your build system for whether
>> you've linked properly against the opencv libs.
>>
>> Jim
>>
>> ______________________________**_________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/**mailman/listinfo/cplusplus-sig<http://mail.python.org/mailman/listinfo/cplusplus-sig>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110921/e7aa46c4/attachment-0001.html>

From talljimbo at gmail.com  Wed Sep 21 17:17:51 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Wed, 21 Sep 2011 11:17:51 -0400
Subject: [C++-sig] Boost Python wrapper for a c++ class that uses Opencv
 2.3
In-Reply-To: <CANY4+HUu4wvU3tEWEBj-mKanWXRbo+3Fh790UuPCa05OROBjAQ@mail.gmail.com>
References: <CANY4+HXpiGpF1V64LbnUvtN4h4aveewNuvfDp-QihemFoX_vtg@mail.gmail.com>
	<CANY4+HWb1nePRZn+k0+bmpB9BSZ5yqzU6yHwZqRv2-kbF5qBrw@mail.gmail.com>
	<4E79EA1D.1010405@gmail.com>
	<CANY4+HX46dPS4RFcUN5PNzCduEcRrnT=EzXQjbGXfyG-r+JFvQ@mail.gmail.com>
	<4E79F78A.9080908@gmail.com>
	<CANY4+HXWx1UNJCm3c_VFrtDKQi9_6suLoBHWbAwV7aPsRP6i9w@mail.gmail.com>
	<CANY4+HUu4wvU3tEWEBj-mKanWXRbo+3Fh790UuPCa05OROBjAQ@mail.gmail.com>
Message-ID: <4E7A001F.5020103@gmail.com>

On 09/21/2011 11:13 AM, Kevin Hughes wrote:
> Here is the JamRoot file I have been using:
>
>     using python ;
>
>     # Specify that the boost-python library exists under the name
>     # boost_python. That is, because the library was installed at the
>     # standard search path as /usr/lib/libboost_python.so, bjam will find
>     # it automatically. No need to specify the absolute path.
>     # lib libboost_python : : <name>boost_python-mt ;
>     lib libboost_python : : <name>boost_python ;
>     lib libboost_date_time : : <name>boost_date_time ;
>
>     # Set up the project-wide requirements that everything uses the
>     # boost_python library.
>     project
>          : requirements <library>libboost_python
>     ;
>
>     # Declare the extension modules.  You can specify multiple
>     # source files after the colon separated by spaces.
>     python-extension libFoo : wrap_foo.cpp ;
>

As it turns out, I've never used bjam, except to build Boost itself.  I 
suspect you need another "requirements" line with the opencv libraries, 
but I'm not sure of the syntax.  Hopefully someone else can help you 
with that.

There's no requirement that Boost.Python projects use bjam, however, and 
if you're already more comfortable with another build system you might 
find it easier to use that.  Python's own distutils can work well for 
simple modules as well.

Jim

From wichert at wiggy.net  Thu Sep 22 15:43:52 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Thu, 22 Sep 2011 15:43:52 +0200
Subject: [C++-sig] map_indexing_suite does not find to-python convertors
Message-ID: <4E7B3B98.9090606@wiggy.net>

I am struggling to expose a map to python code. What I am doing is 
pretty simple: I have a map from an enum type to Glib::ustring, with a 
convertors registered to convert Python str and unicode instances to an 
ustring, and ustring to Python unicode. I've put the relevant code below.

What I am running into is that the conversion from python to C++ types 
is failing with a "No Python class registered for C++ class 
Glib::ustring" error. This looks very similar to the FAQ entry "Why is 
my automatic to-python conversion not being found?" 
(http://www.boost.org/doc/libs/1_47_0/libs/python/doc/v2/faq.html#topythonconversionfailed 
), and the workaround suggested there indeed works fine for class 
properties. I can't seem to find a similar trick for map values though. 
Is there a trick to get this working?

Wichert.



enum aspect_type { ... };
typedef std::map<aspect_type, Glib::ustring>  aspect_map;

struct ustring_to_python {
	static PyObject* convert(Glib::ustring const&  s) {
		PyObject* unicode;
		unicode=PyUnicode_DecodeUTF8(s.data(), static_cast<Py_ssize_t>(s.size()), "ignore");
		Py_INCREF(unicode);
		return unicode;
	}
};

struct ustring_from_python {
	ustring_from_python() {
		converter::registry::push_back(&convertible,&construct, type_id<Glib::ustring>());
	}

	static void* convertible(PyObject *obj) {
		if (PyUnicode_Check(obj) || PyString_Check(obj))
			return obj;
		return 0;
	}

	static void construct(PyObject* obj, converter::rvalue_from_python_stage1_data *data) {
		PyObject *str = 0;
		char* value;

		if (PyUnicode_Check(obj)) {
			str = PyUnicode_AsUTF8String(obj);
			if (str==0)
				throw_error_already_set();
		}
		
		value = PyString_AsString(obj);
		if (value==0) {
			Py_XDECREF(str);
			throw_error_already_set();
		}

		void* storage = ((converter::rvalue_from_python_storage<Glib::ustring>*)data)->storage.bytes;
		new (storage) Glib::ustring(value);
		data->convertible = storage;
		Py_XDECREF(obj);
	}
};



BOOST_PYTHON_MODULE(mymodule) {
     to_python_converter<Glib::ustring, ustring_to_python>();
     ustring_from_python();

     class_<Article::aspect_map>("AspectMap")
         .def(map_indexing_suite<aspect_map, true>());


}



From talljimbo at gmail.com  Thu Sep 22 16:31:54 2011
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 22 Sep 2011 10:31:54 -0400
Subject: [C++-sig] map_indexing_suite does not find to-python convertors
In-Reply-To: <4E7B3B98.9090606@wiggy.net>
References: <4E7B3B98.9090606@wiggy.net>
Message-ID: <4E7B46DA.5090105@gmail.com>

On 09/22/2011 09:43 AM, Wichert Akkerman wrote:
> I am struggling to expose a map to python code. What I am doing is
> pretty simple: I have a map from an enum type to Glib::ustring, with a
> convertors registered to convert Python str and unicode instances to an
> ustring, and ustring to Python unicode. I've put the relevant code below.
>
> What I am running into is that the conversion from python to C++ types
> is failing with a "No Python class registered for C++ class
> Glib::ustring" error. This looks very similar to the FAQ entry "Why is
> my automatic to-python conversion not being found?"
> (http://www.boost.org/doc/libs/1_47_0/libs/python/doc/v2/faq.html#topythonconversionfailed
> ), and the workaround suggested there indeed works fine for class
> properties. I can't seem to find a similar trick for map values though.
> Is there a trick to get this working?
>

I think the easiest solution would be to try to switch to indexing suite 
v2 (I think the newest version is found in the Py++ sources, although 
you don't have to use Py++ to make use of it).  With that, you can set 
the call policies for returning elements from your container.

There are ways to do this with indexing suite v1 (the version that comes 
with Boost.Python), but it'd be a lot harder, and I'm not certain what 
approach to recommend that you try first in that case.


Jim

From wichert at wiggy.net  Thu Sep 22 16:50:46 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Thu, 22 Sep 2011 16:50:46 +0200
Subject: [C++-sig] map_indexing_suite does not find to-python convertors
In-Reply-To: <4E7B46DA.5090105@gmail.com>
References: <4E7B3B98.9090606@wiggy.net> <4E7B46DA.5090105@gmail.com>
Message-ID: <4E7B4B46.8050808@wiggy.net>

On 09/22/2011 04:31 PM, Jim Bosch wrote:
> On 09/22/2011 09:43 AM, Wichert Akkerman wrote:
>> I am struggling to expose a map to python code. What I am doing is
>> pretty simple: I have a map from an enum type to Glib::ustring, with a
>> convertors registered to convert Python str and unicode instances to an
>> ustring, and ustring to Python unicode. I've put the relevant code 
>> below.
>>
>> What I am running into is that the conversion from python to C++ types
>> is failing with a "No Python class registered for C++ class
>> Glib::ustring" error. This looks very similar to the FAQ entry "Why is
>> my automatic to-python conversion not being found?"
>> (http://www.boost.org/doc/libs/1_47_0/libs/python/doc/v2/faq.html#topythonconversionfailed 
>>
>> ), and the workaround suggested there indeed works fine for class
>> properties. I can't seem to find a similar trick for map values though.
>> Is there a trick to get this working?
>>
>
> I think the easiest solution would be to try to switch to indexing 
> suite v2 (I think the newest version is found in the Py++ sources, 
> although you don't have to use Py++ to make use of it).  With that, 
> you can set the call policies for returning elements from your container.

Is 
http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ 
the latest version of that? I'll give it a try.

wichert.


From wichert at wiggy.net  Thu Sep 22 17:25:02 2011
From: wichert at wiggy.net (Wichert Akkerman)
Date: Thu, 22 Sep 2011 17:25:02 +0200
Subject: [C++-sig] map_indexing_suite does not find to-python convertors
In-Reply-To: <4E7B4B46.8050808@wiggy.net>
References: <4E7B3B98.9090606@wiggy.net> <4E7B46DA.5090105@gmail.com>
	<4E7B4B46.8050808@wiggy.net>
Message-ID: <4E7B534E.8000009@wiggy.net>

On 09/22/2011 04:50 PM, Wichert Akkerman wrote:
> On 09/22/2011 04:31 PM, Jim Bosch wrote:
>> On 09/22/2011 09:43 AM, Wichert Akkerman wrote:
>>> I am struggling to expose a map to python code. What I am doing is
>>> pretty simple: I have a map from an enum type to Glib::ustring, with a
>>> convertors registered to convert Python str and unicode instances to an
>>> ustring, and ustring to Python unicode. I've put the relevant code 
>>> below.
>>>
>>> What I am running into is that the conversion from python to C++ types
>>> is failing with a "No Python class registered for C++ class
>>> Glib::ustring" error. This looks very similar to the FAQ entry "Why is
>>> my automatic to-python conversion not being found?"
>>> (http://www.boost.org/doc/libs/1_47_0/libs/python/doc/v2/faq.html#topythonconversionfailed 
>>>
>>> ), and the workaround suggested there indeed works fine for class
>>> properties. I can't seem to find a similar trick for map values though.
>>> Is there a trick to get this working?
>>>
>>
>> I think the easiest solution would be to try to switch to indexing 
>> suite v2 (I think the newest version is found in the Py++ sources, 
>> although you don't have to use Py++ to make use of it).  With that, 
>> you can set the call policies for returning elements from your 
>> container.
>
> Is 
> http://pygccxml.svn.sourceforge.net/viewvc/pygccxml/pyplusplus_dev/indexing_suite_v2/ 
> the latest version of that? I'll give it a try.

That certainly seems to solve the conversion problem I have been seeing. 
It does have some flaws though, such as find_or_throw  raising a 
ValueError instead of a KeyError when a key is not found in a mapping.

Is there an active bug tracker for the indexing suite where this could 
be filed?

Wichert.



From k.hughes at queensu.ca  Fri Sep 23 22:34:12 2011
From: k.hughes at queensu.ca (Kevin Hughes)
Date: Fri, 23 Sep 2011 16:34:12 -0400
Subject: [C++-sig] Boost Python class won't import
Message-ID: <CANY4+HX3oJYukOyHYTc2CyAzhVP4Ejkr3_Hsw9a+6Rcfj2GoDg@mail.gmail.com>

Hello,

I am trying to wrap a simple class that runs some image processing routines.
I would like them to be script-able from python.

I am not trying to pass complicated types, in fact almost all the functions
take and return no arguments. Here is my wrapper code (generated with
pyplusplus partially):


#include "boost/python.hpp"

#include "eventDetector.h"

namespace bp = boost::python;

BOOST_PYTHON_MODULE(pyplusplus){

    { //::eventDetector
        typedef bp::class_< eventDetector > eventDetector_exposer_t;
        eventDetector_exposer_t eventDetector_exposer =
eventDetector_exposer_t( "eventDetector", bp::init< params & >((
bp::arg("op") )) );
        bp::scope eventDetector_scope( eventDetector_exposer );
        bp::implicitly_convertible< params &, eventDetector >();
        { //::eventDetector::detect

            typedef void ( ::eventDetector::*detect_function_type )(  ) ;

            eventDetector_exposer.def(
                "detect"
                , detect_function_type( &::eventDetector::detect ) );

        }
        { //::eventDetector::init

            typedef void ( ::eventDetector::*init_function_type )(  ) ;

            eventDetector_exposer.def(
                "init"
                , init_function_type( &::eventDetector::init ) );

        }
        { //::eventDetector::load

            typedef void ( ::eventDetector::*load_function_type )(  ) ;

            eventDetector_exposer.def(
                "load"
                , load_function_type( &::eventDetector::load ) );

        }
        { //::eventDetector::save

            typedef void ( ::eventDetector::*save_function_type )(  ) ;

            eventDetector_exposer.def(
                "save"
                , save_function_type( &::eventDetector::save ) );

        }
        { //::eventDetector::setGridSize

            typedef void ( ::eventDetector::*setGridSize_function_type )(
int,int ) ;

            eventDetector_exposer.def(
                "setGridSize"
                , setGridSize_function_type( &::eventDetector::setGridSize )
                , ( bp::arg("x"), bp::arg("y") ) );

        }
        { //::eventDetector::setVideoCapture

            typedef bool ( ::eventDetector::*setVideoCapture_function_type
)( ::std::string ) ;

            eventDetector_exposer.def(
                "setVideoCapture"
                , setVideoCapture_function_type(
&::eventDetector::setVideoCapture )
                , ( bp::arg("filename") ) );

        }
        { //::eventDetector::setVideoCapture

            typedef bool ( ::eventDetector::*setVideoCapture_function_type
)( int ) ;

            eventDetector_exposer.def(
                "setVideoCapture"
                , setVideoCapture_function_type(
&::eventDetector::setVideoCapture )
                , ( bp::arg("cam") ) );

        }
        { //::eventDetector::train

            typedef void ( ::eventDetector::*train_function_type )(  ) ;

            eventDetector_exposer.def(
                "train"
                , train_function_type( &::eventDetector::train ) );

        }
    }

    bp::class_< params >( "params" )
        ... large list of simple params...;

}


It builds fine using bjam but when I import into python I get an error
message: libEventDetector.so: undefined symbol: _ZN13eventDetector4loadEv.

I am really not very good with boost::python and I don't really know what to
do. Help is appreciated, thank you!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110923/244f0d28/attachment.html>

From stefan at seefeld.name  Fri Sep 23 23:04:19 2011
From: stefan at seefeld.name (Stefan Seefeld)
Date: Fri, 23 Sep 2011 17:04:19 -0400
Subject: [C++-sig] Boost Python class won't import
In-Reply-To: <CANY4+HX3oJYukOyHYTc2CyAzhVP4Ejkr3_Hsw9a+6Rcfj2GoDg@mail.gmail.com>
References: <CANY4+HX3oJYukOyHYTc2CyAzhVP4Ejkr3_Hsw9a+6Rcfj2GoDg@mail.gmail.com>
Message-ID: <4E7CF453.3060800@seefeld.name>

On 09/23/2011 04:34 PM, Kevin Hughes wrote:
>
> It builds fine using bjam but when I import into python I get an error
> message: libEventDetector.so: undefined symbol: _ZN13eventDetector4loadEv.
>
> I am really not very good with boost::python and I don't really know
> what to do. Help is appreciated, thank you!

The problem is unrelated to boost.python. It appears you are not linking
all the symbols that are required. Try to find where the
"_ZN13eventDetector4loadEv" symbol (in C++: "eventDetector::load()") is
defined, and what object file or library you need to link to when
building your extension module.

HTH,
        Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From k.hughes at queensu.ca  Wed Sep 28 19:45:18 2011
From: k.hughes at queensu.ca (Kevin Hughes)
Date: Wed, 28 Sep 2011 13:45:18 -0400
Subject: [C++-sig] How to Tell Bjam about a Static Lib it needs to build the
	extension
Message-ID: <CANY4+HW-rK4OMn6mPm7i96CNLoCE6ZqFkcOBE43QXVZmG19JPA@mail.gmail.com>

Hi,

I think I am getting very close to successfully using boost python for the
first time. All I need now is to tell Bjam about a static library my program
depends on. I can't figure out how to do this in my JamRoot file.

Thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20110928/1893b08d/attachment.html>

From dave at boostpro.com  Thu Sep 29 00:35:14 2011
From: dave at boostpro.com (Dave Abrahams)
Date: Wed, 28 Sep 2011 18:35:14 -0400
Subject: [C++-sig] How to Tell Bjam about a Static Lib it needs to build
	the extension
References: <CANY4+HW-rK4OMn6mPm7i96CNLoCE6ZqFkcOBE43QXVZmG19JPA@mail.gmail.com>
Message-ID: <m2bou4gusd.fsf@pluto.luannocracy.com>


on Wed Sep 28 2011, Kevin Hughes <k.hughes-AT-queensu.ca> wrote:

> Hi,
>
> I think I am getting very close to successfully using boost python
> for the first time. All I need now is to tell Bjam about a static
> library my program depends on. I can't figure out how to do this in
> my JamRoot file.

I suggest you try the Boost Build mailing list for this question if you
don't get an answer here.

Cheers,

-- 
Dave Abrahams
BoostPro Computing
http://www.boostpro.com


From avibahra at googlemail.com  Thu Sep 29 12:17:08 2011
From: avibahra at googlemail.com (Avi Bahra)
Date: Thu, 29 Sep 2011 11:17:08 +0100
Subject: [C++-sig] bjam: How to determine python version for installation ?
Message-ID: <CAHwHRFH3O62N+87UQYMHmziRoJpOp0TyDfMXNZhm3DoiXG9G0A@mail.gmail.com>

As a part of installation of my extension I need determine the
python extension ( without having to hard code it)

Currently you can specify the python version, but I need a way
of determining which python version is being used, when we
specify:  'using python ;'

Is this possible ?

   Best regards,
Ta,
    Avi