From brokenn at gmail.com  Wed Sep  1 09:41:13 2010
From: brokenn at gmail.com (Brian O'Kennedy)
Date: Wed, 1 Sep 2010 08:41:13 +0100
Subject: [C++-sig] question about object
In-Reply-To: <AANLkTi=nZUfeCikj+urtU-Hi4jiMOdHQZp0ZuUOUS8w1@mail.gmail.com>
References: <AANLkTi=nZUfeCikj+urtU-Hi4jiMOdHQZp0ZuUOUS8w1@mail.gmail.com>
Message-ID: <AANLkTi=4pQvqaPmZEVT0TF1xaixTi8VOpmg4z3B9yqc-@mail.gmail.com>

I'm guessing it's the boost python equivalent of the Python code:
>>> "%s is bigger than %s" % (NAME,name)

The c++ statement is doing the same, via operator overloading. If we look at
the statement bit-by-bit:

object msg="%s is bigger than %s" % make_tuple(NAME,name);
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Creating a boost python object called 'msg' and assign the string "%s is
bigger than %s" to it.

object msg="%s is bigger than %s" % make_tuple(NAME,name);
                                                    ^^^^
The '%' operator is overloaded by boost python to work the same as in
python.

object msg="%s is bigger than %s" % make_tuple(NAME,name);

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Create a tuple of NAME,name and pass that to the '%' operator.

Cheers.
 brian


On 31 August 2010 22:46, Junwei Zhang <junweizhang2006 at gmail.com> wrote:

> Hi all,
>
> who can tell me what c++ semantics of following  statement.
> I do not understand it, it do not like usual c++ codes
>
> object msg="%s is bigger than %s" %make_tuple(NAME,name);
>
> --
> Junwei Zhang
> Office Phone:402-472-1968
> junweizhang2006 at gmail.com
> www.junweizhang.com
> Department of Electrical Engineering
> University of Nebraska-Lincoln
> 209N Walter Scott Engineering Center
> P.O. Box 880511
> Lincoln, NE 68588-0511
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100901/2ae028f6/attachment.html>

From David.Aldrich at EU.NEC.COM  Thu Sep  2 12:36:41 2010
From: David.Aldrich at EU.NEC.COM (David Aldrich)
Date: Thu, 2 Sep 2010 11:36:41 +0100
Subject: [C++-sig] Help needed with Boost Python 'Hello world' tutorial
Message-ID: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1C3E@EUEXCLU01.EU.NEC.COM>

Hi

I am using Boost 1.43 on Windows XP, with MSVC 9.0 and Python 2.7.

I am trying to run the 'Hello world' example in libs\python\example\tutorial

I have created  a user-config.jam file containing:

       # MSVC configuration
       using msvc : 9.0 ;

       # Python configuration
       using python : 2.7 : C:/Python27 ;

I then execute:

       C:\boost_1_43_0\libs\python\example\tutorial>\boost_1_43_0\tools\jam\src\bin.ntx86\bjam.exe

The build succeeds and hello.test passes, but I have two questions:

1) hello_ext is built as a static library: hello_ext.lib. From the tutorial, I was expecting a dll. Is a static build correct? 

2) How may I run the test from the command line?

Best regards

David


From macieksitarz at wp.pl  Sat Sep  4 14:14:36 2010
From: macieksitarz at wp.pl (Maciej Sitarz)
Date: Sat, 04 Sep 2010 14:14:36 +0200
Subject: [C++-sig] Bindings compilation with bjam. release build doesn't work
Message-ID: <4C82382C.2070301@wp.pl>

Hi,
I have generated python binding with pyplusplus, I was building it all 
the time with bjam using debug build type. Everythin worked fine.

Now when I tried to build it in release mode, importing the module fails 
with "Segmentation fault" message.

$ PYTHONPATH="./bin/gcc-4.1.2/release" python26
Python 2.6.5 (r265:79063, Apr  9 2010, 11:16:46)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
 >>> import Cream
Segmentation fault

I'm using:
- gcc 4.1.2
- boost 1.44
- python 2.6.5
- gccxml 0.9( 1.135 )
- pygccxml and pyplusplus from SVN

The difference betweend debug and relase shows during compilation. 
There's an additional warning message when compiling in release:

/home/macieks/boost/boost_1_44_0/boost/python/converter/pyobject_type.hpp: 
In static member function 'static bool 
boost::python::converter::pyobject_type<Object, 
pytype>::check(PyObject*) [with Object = boost::python::indexing::slice, 
PyTypeObject* pytype = (& PySlice_Type)]':
/home/macieks/boost/boost_1_44_0/boost/python/converter/obj_mgr_arg_from_python.hpp:69: 
   instantiated from 'bool 
boost::python::converter::object_manager_value_arg_from_python<T>::convertible() 
const [with T = boost::python::indexing::slice]'
/home/macieks/boost/boost_1_44_0/boost/preprocessor/iteration/detail/local.hpp:37: 
   instantiated from 'PyObject* 
boost::python::detail::caller_arity<2u>::impl<F, Policies, 
Sig>::operator()(PyObject*, PyObject*) [with F = void 
(*)(std::vector<CREAMTYPES__Property*, 
std::allocator<CREAMTYPES__Property*> >&, 
boost::python::indexing::slice), Policies = 
boost::python::return_internal_reference<1ul, 
boost::python::default_call_policies>, Sig = boost::mpl::vector3<void, 
std::vector<CREAMTYPES__Property*, std::allocator<CREAMTYPES__Property*> 
 >&, boost::python::indexing::slice>]'
/home/macieks/boost/boost_1_44_0/boost/python/object/py_function.hpp:38: 
   instantiated from 'PyObject* 
boost::python::objects::caller_py_function_impl<Caller>::operator()(PyObject*, 
PyObject*) [with Caller = boost::python::detail::caller<void 
(*)(std::vector<CREAMTYPES__Property*, 
std::allocator<CREAMTYPES__Property*> >&, 
boost::python::indexing::slice), 
boost::python::return_internal_reference<1ul, 
boost::python::default_call_policies>, boost::mpl::vector3<void, 
std::vector<CREAMTYPES__Property*, std::allocator<CREAMTYPES__Property*> 
 >&, boost::python::indexing::slice> >]'
Cream.cpp:818:   instantiated from here
/home/macieks/boost/boost_1_44_0/boost/python/converter/pyobject_type.hpp:21: 
warning: dereferencing type-punned pointer will break strict-aliasing rules

relase build messages: http://pastie.org/1137650
debug build messages: http://pastie.org/1137649
Jamroot file: http://pastie.org/1137763

Thanks for help.
Cheers,
-- 
Maciek Sitarz

From macieksitarz at wp.pl  Sat Sep  4 18:47:51 2010
From: macieksitarz at wp.pl (Maciej Sitarz)
Date: Sat, 04 Sep 2010 18:47:51 +0200
Subject: [C++-sig] Bindings compilation with bjam. release build doesn't
 work
In-Reply-To: <4C82382C.2070301@wp.pl>
References: <4C82382C.2070301@wp.pl>
Message-ID: <4C827837.5020204@wp.pl>

I forgot to add the file I'm compiling:

Cream.cpp: http://pastie.org/1137778

Cheers
-- 
Maciek Sitarz

From David.Aldrich at EU.NEC.COM  Mon Sep  6 17:15:39 2010
From: David.Aldrich at EU.NEC.COM (David Aldrich)
Date: Mon, 6 Sep 2010 16:15:39 +0100
Subject: [C++-sig] Advice sought on making a large C++ application a Python
	extension
Message-ID: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>

Hi

We have a large C++ application that we develop in-house. It consists of a large number of source files, some of which are linked directly and some of which are first built as static libraries.

It has been suggested that we make the application's functionality available as a Python extension.  Simplistically speaking, we could then let users replace our C++ main() by a Python script.

I have only looked at the boost.python tutorial that demonstrates a 'Hello world' Python extension.  For our project we would be dealing with something very much bigger, with many classes, singleton classes and (possibly) global data.  My question is, would it be practical to make all of those entities Python extensions?

We would also want to continue to support the application as a wholly C++ application.   Currently, we use gnu make to build it.  Would I have to add a parallel build system using bjam to build the extensions? And then maintain both make's makefiles and bjam's Jamroot files?

Can .pyd files be loaded by the C++ linker?

Any advice would be gratefully received.

Best regards

David

=================================================================
David Aldrich, NEC Telecom MODUS, Ltd,
Cleeve Road, Leatherhead, Surrey, KT22 7SA, UK
Direct tel. +44 (0) 1372 381857
=================================================================


From lists_ravi at lavabit.com  Mon Sep  6 18:03:38 2010
From: lists_ravi at lavabit.com (Ravi)
Date: Mon, 6 Sep 2010 09:03:38 -0700
Subject: [C++-sig] Advice sought on making a large C++ application a
	Python extension
In-Reply-To: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
Message-ID: <201009060903.39059.lists_ravi@lavabit.com>

On Monday 06 September 2010 08:15:39 David Aldrich wrote:
> I have only looked at the boost.python tutorial that demonstrates a 'Hello
> world' Python extension.  For our project we would be dealing with
> something very much bigger, with many classes, singleton classes and
> (possibly) global data.  My question is, would it be practical to make all
> of those entities Python extensions?

Yes. PyOGRE or cctbx are excellent exmples of larger projects which use 
boost.python. You may want to use py++ to automate binding generation to save 
yourself quite some time.
 
> We would also want to continue to support the application as a wholly C++
> application.   Currently, we use gnu make to build it.  Would I have to
> add a parallel build system using bjam to build the extensions? And then
> maintain both make's makefiles and bjam's Jamroot files?

No. The python modules can be compiled using regular make. My recommendation 
is to avoid using bjam, even if it is technically superior, as it is not used 
widely enough to have the critical mass that other tools (autotools, cmake, 
scons) have which means that finding help is pretty painful (as there seem to 
be only a couple of people with knowledge sufficient to answer any non-trivial 
questions).

> Can .pyd files be loaded by the C++ linker?

These are simply shared libraries (on POSIX). So the linker can read them. 
Avoid storing the python bindings in the same shared libraries as your core 
code since the coupling will be hard to undo while gaining nothing.

Regards,
Ravi


From David.Aldrich at EU.NEC.COM  Mon Sep  6 18:50:45 2010
From: David.Aldrich at EU.NEC.COM (David Aldrich)
Date: Mon, 6 Sep 2010 17:50:45 +0100
Subject: [C++-sig] Advice sought on making a large C++ application a
 Python extension
In-Reply-To: <201009060903.39059.lists_ravi@lavabit.com>
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
	<201009060903.39059.lists_ravi@lavabit.com>
Message-ID: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1E08@EUEXCLU01.EU.NEC.COM>

Hi Ravi

Thanks for your reply.

> Yes. PyOGRE or cctbx are excellent exmples of larger projects which use
> boost.python. You may want to use py++ to automate binding generation to
> save yourself quite some time.

Ok, I will look at py++.

> No. The python modules can be compiled using regular make. My
> recommendation
> is to avoid using bjam, even if it is technically superior, as it is not
> used
> widely enough to have the critical mass that other tools (autotools, cmake,
> scons) have which means that finding help is pretty painful (as there seem
> to
> be only a couple of people with knowledge sufficient to answer any non-
> trivial
> questions).

Understood. Though the boost community seems to promote bjam quite hard ;-)

> These are simply shared libraries (on POSIX). So the linker can read them.
> Avoid storing the python bindings in the same shared libraries as your
> core code since the coupling will be hard to undo while gaining nothing.

So, I should make both normal dll's for the C++ application and .pyd extensions for the Python interface?

Best regards

David

From benoit-l at moving-picture.com  Mon Sep  6 20:08:58 2010
From: benoit-l at moving-picture.com (Benoit Leveau)
Date: Mon, 06 Sep 2010 19:08:58 +0100
Subject: [C++-sig] Advice sought on making a large C++ application a
 Python extension
In-Reply-To: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1E08@EUEXCLU01.EU.NEC.COM>
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>	<201009060903.39059.lists_ravi@lavabit.com>
	<0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1E08@EUEXCLU01.EU.NEC.COM>
Message-ID: <4C852E3A.1090501@moving-picture.com>

David Aldrich wrote:
> Hi Ravi
> 
> Thanks for your reply.
> 
>> Yes. PyOGRE or cctbx are excellent exmples of larger projects which use
>> boost.python. You may want to use py++ to automate binding generation to
>> save yourself quite some time.
> 
> Ok, I will look at py++.
py++ is very good for the goal you have in mind, but you should start to get your hands on it
with something small. I recommend that you first try to bind only one of your libraries,
and develop a framework to do that bindng (some classes/scripts that contain py++ code you'll use a
lot, like adding documentation to a function, excluding functions that have warnings, etc.)
Else, you'll end up having a big python script for each library you want to bind, whereas with this approach
you can use the same "binding generator" for all your libraries.

If you write some command line scripts that generate the binding, you can finally - directly from
your Makefile - generate and compile the binding code.

>> No. The python modules can be compiled using regular make. My
>> recommendation
>> is to avoid using bjam, even if it is technically superior, as it is not
>> used
>> widely enough to have the critical mass that other tools (autotools, cmake,
>> scons) have which means that finding help is pretty painful (as there seem
>> to
>> be only a couple of people with knowledge sufficient to answer any non-
>> trivial
>> questions).
> 
> Understood. Though the boost community seems to promote bjam quite hard ;-)
To compile the boost libraries (including boost python), it is indeed best to use bjam
(especially considering that you generally don't have to customize that).
But, then to link your project against these boost libraries, you have no real benefit
of using bjam IMHO.
 
>> These are simply shared libraries (on POSIX). So the linker can read them.
>> Avoid storing the python bindings in the same shared libraries as your
>> core code since the coupling will be hard to undo while gaining nothing.
> 
> So, I should make both normal dll's for the C++ application and .pyd extensions for the Python interface?

Yes, and actually your pyd libraries will link against these dlls, so:
- the pyd files contain only the python-to/from-c++ layer
- whereas your dlls contain your original application code

Cheers,
Benoit

From binbinsh at gmail.com  Tue Sep  7 09:19:40 2010
From: binbinsh at gmail.com (Binbin Shen)
Date: Tue, 7 Sep 2010 15:19:40 +0800
Subject: [C++-sig] Some questions when making a C++ project as python
	externsions
Message-ID: <AANLkTim4k+w1S9qijUaw1V0sYRiXo6nU8gsE4vNd=2-w@mail.gmail.com>

Hi,
I am new to boost.python, and want to provide python interface of a
large lab application written in C++. I have written some wrappers and
make some classes works in python, but also encounter some problems
that cannot be solved by Googling, and I do not want to modify the
source of the C++ application, some of the problems are as follows:

1. Some C++ functions have return the result by parameters like this:
void add(std::wstring& a, const wstring b) {
?? ?a+=b;
}
>>> a = unicode()
>>> b = u"test"
>>> add(a, b) ? ? ? ? ? ? ? # error when compiling, unicode can not convert to wstring automatically

I should pass a python unicode into this function and return by "a" implicitly.

2. Some C++ functions show pass a "void*" pointer that pointing a
pre-allocated memory into the function, like this:
void read(void *memory, int size, int count)
{
?? ? ?// copy something into memory
}

How can I make a void pointer in python and pass it to read()?

3. Some virtual function with default arguments, like this:
class foo {
public:
?? ?virtual int add(int x, int y = 10) { return x+y; }
};
and the wrapper looks like:
struct fooWrap {
?? ?int add(int x, int y) {
?? ? ? ?if (override add_ = this->get_override("add")
?? ? ? ? ? ?return add_(x, y);
?? ? ? ?return this->foo::add(x, y);
?? ?}
?? ?int default add(int x, int y) {
?? ? ? ?return this->foo::add(x, y);
?? ?}
}
and
??.def("add", &foo::add, &fooWrap::default_add);
but when dealing with overloads, it looks like:
BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(add_overloads, add, 1, 2)
and
?? .def("add", &foo::add, add_overloads())

How can I write these two together?

4. Some functions have variable length parameter:
std::wstring format(const wchar_t* strFormat, ...) {
?? ?// use va_list, va_start, va_end...
}

how boost.python deal with it?

5. The last question is that I do not quiet understand the policies,
:-(, is there any detail documents introduce them:
with_custodian_and_ward:?Ties lifetimes of the arguments
with_custodian_and_ward_postcall:?Ties lifetimes of the arguments and results
return_internal_reference:?Ties lifetime of one argument to that of result
return_value_policy<T> with T one of:
reference_existing_object:?naive (dangerous) approach object
copy_const_reference: Boost.Python v1 approach
copy_non_const_reference:
manage_new_object: Adopt a pointer and hold the instance
return_by_value
return_by_reference
return_opaque_pointer

I use boost 1.42 with gcc 4.4.3 in Gentoo Linux.

Any advice would be gratefully received.

Best regards

Binbin

From talljimbo at gmail.com  Tue Sep  7 10:05:10 2010
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 07 Sep 2010 01:05:10 -0700
Subject: [C++-sig] Some questions when making a C++ project as python
 externsions
In-Reply-To: <AANLkTim4k+w1S9qijUaw1V0sYRiXo6nU8gsE4vNd=2-w@mail.gmail.com>
References: <AANLkTim4k+w1S9qijUaw1V0sYRiXo6nU8gsE4vNd=2-w@mail.gmail.com>
Message-ID: <4C85F236.2040804@gmail.com>

On 09/07/2010 12:19 AM, Binbin Shen wrote:
> Hi,
> I am new to boost.python, and want to provide python interface of a
> large lab application written in C++. I have written some wrappers and
> make some classes works in python, but also encounter some problems
> that cannot be solved by Googling, and I do not want to modify the
> source of the C++ application, some of the problems are as follows:
>

Here's an obvious-only-once-you've-seen-it trick that applies to a lot 
of your questions:  if a C++ function isn't in a form where you can wrap 
it with Boost.Python, write a new C++ function that calls it and wrap 
that.  This even works for member functions; if the first argument of a 
free function is a class type, it can be wrapped in Python as a member 
function of that class.


> 1. Some C++ functions have return the result by parameters like this:
> void add(std::wstring&  a, const wstring b) {
>      a+=b;
> }
>>>> a = unicode()
>>>> b = u"test"
>>>> add(a, b)               # error when compiling, unicode can not convert to wstring automatically
>
> I should pass a python unicode into this function and return by "a" implicitly.
>

Unicode may be a little tricky; I don't think Boost.Python handles it at 
all automatically.  You may have to use the Python C-API a bit (others 
on this list may know more).  You'll probably want to make one argument 
of your C++ wrapper function a boost::python::object; you can then 
extract a PyObject* with the .ptr() member function 
(boost::python::object is basically a fancy smart pointer for 
PyObject*).  You can then return a boost::python::object containing a 
real Python unicode object.

Without the unicode part, turning a result-by-arg function into a 
result-by-return is of course easy to solve with a one-line C++ wrapper 
function (and one of the wrapper-generators, Py++, can do this 
automatically for you in some cases).


> 2. Some C++ functions show pass a "void*" pointer that pointing a
> pre-allocated memory into the function, like this:
> void read(void *memory, int size, int count)
> {
>        // copy something into memory
> }
>
> How can I make a void pointer in python and pass it to read()?
>

That depends on what you want that void pointer points to.  Is it a 
wrapped C++ class, something that maps to a Python built-in type (or a 
numpy array), or is it just raw, unallocated memory?

What do you want to do with it in Python?  Just pass it to another 
wrapped C++ function?

There are solutions for all of these, but they all require some work.


> 3. Some virtual function with default arguments, like this:
> class foo {
> public:
>      virtual int add(int x, int y = 10) { return x+y; }
> };
> and the wrapper looks like:
> struct fooWrap {
>      int add(int x, int y) {
>          if (override add_ = this->get_override("add")
>              return add_(x, y);
>          return this->foo::add(x, y);
>      }
>      int default add(int x, int y) {
>          return this->foo::add(x, y);
>      }
> }
> and
>    .def("add",&foo::add,&fooWrap::default_add);
> but when dealing with overloads, it looks like:
> BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(add_overloads, add, 1, 2)
> and
>     .def("add",&foo::add, add_overloads())
>
> How can I write these two together?

Not sure; I just haven't used the OVERLOADS macros much.  Hopefully 
someone else can answer.

>
> 4. Some functions have variable length parameter:
> std::wstring format(const wchar_t* strFormat, ...) {
>      // use va_list, va_start, va_end...
> }
>
> how boost.python deal with it?
>

It can't directly.  You'll have to write a C++ wrapper function.  You 
may want to look into boost::python::raw_function (see the reference 
documentation), which lets you write a Python function with a variable 
number of arguments.


> 5. The last question is that I do not quiet understand the policies,
> :-(, is there any detail documents introduce them:
> with_custodian_and_ward: Ties lifetimes of the arguments
> with_custodian_and_ward_postcall: Ties lifetimes of the arguments and results
> return_internal_reference: Ties lifetime of one argument to that of result
> return_value_policy<T>  with T one of:
> reference_existing_object: naive (dangerous) approach object
> copy_const_reference: Boost.Python v1 approach
> copy_non_const_reference:
> manage_new_object: Adopt a pointer and hold the instance
> return_by_value
> return_by_reference
> return_opaque_pointer
>
> I use boost 1.42 with gcc 4.4.3 in Gentoo Linux.
>
> Any advice would be gratefully received.
>

The ones you'll probably use most often are:

- return_internal_reference: Main use is when a class returns a pointer 
or reference to a data member.  It keeps the Python object wrapping the 
class instance alive as long as the Python object wrapping the data 
member is alive.

- return_value_policy + copy_const_reference or 
copy_non_const_reference: Instead of wrapping a reference, invoke the 
copy constructor and return the new object.

- manage_new_object: assumes the return value was allocated by the new 
operator and invokes delete when Python is done with it.

When you return by value or smart pointer, there's generally no need to 
use call policies, and Boost.Python is prettying good at throwing up a 
compile error when you need to use them.


Hope that helps!

Jim Bosch

From bfitzpatrick at vtiinstruments.com  Tue Sep  7 16:45:53 2010
From: bfitzpatrick at vtiinstruments.com (Ben Fitzpatrick)
Date: Tue, 7 Sep 2010 09:45:53 -0500
Subject: [C++-sig] Advice sought on making a large C++ application a
	Python	extension
In-Reply-To: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
Message-ID: <27AD07BD25612C40AF5C78CB49D09B4967DEFF97F9@34093-MBX-C08.mex07a.mlsrvr.com>

David,

I'm still a newbie on this list myself, but I will tell you that for the larger project which we were attempting to wrap, we had excellent luck with Gustavo Carniero's Pybindgen project. We found it to be much simpler to work with than boost.python and py++, and also it compiled much faster.

As to how to make your functionality available with a Python extension - I'm not sure exactly what kind of app you have, but is it possible to make the Python portion only an interface to the underlying C++ code? That's basically what we did with our application - write a wrapper layer on top of the ordinary C++ API that implemented a Python API. This also allowed us to build the C++ application, and then with an additional build step (in GNU make) we were able to compile the python interface layer.

It may be that you need to make all of your classes into python extensions, in which case py++ and boost.python may be the better tool, I am not certain, but for writing just an interface layer, our experience was that Pybindgen was easier.

Hope this helps!

-Ben

-----Original Message-----
From: cplusplus-sig-bounces+bfitzpatrick=vtiinstruments.com at python.org [mailto:cplusplus-sig-bounces+bfitzpatrick=vtiinstruments.com at python.org] On Behalf Of David Aldrich
Sent: Monday, September 06, 2010 11:16 AM
To: Development of Python/C++ integration
Subject: [C++-sig] Advice sought on making a large C++ application a Python extension

Hi

We have a large C++ application that we develop in-house. It consists of a large number of source files, some of which are linked directly and some of which are first built as static libraries.

It has been suggested that we make the application's functionality available as a Python extension.  Simplistically speaking, we could then let users replace our C++ main() by a Python script.

I have only looked at the boost.python tutorial that demonstrates a 'Hello world' Python extension.  For our project we would be dealing with something very much bigger, with many classes, singleton classes and (possibly) global data.  My question is, would it be practical to make all of those entities Python extensions?

We would also want to continue to support the application as a wholly C++ application.   Currently, we use gnu make to build it.  Would I have to add a parallel build system using bjam to build the extensions? And then maintain both make's makefiles and bjam's Jamroot files?

Can .pyd files be loaded by the C++ linker?

Any advice would be gratefully received.

Best regards

David

=================================================================
David Aldrich, NEC Telecom MODUS, Ltd,
Cleeve Road, Leatherhead, Surrey, KT22 7SA, UK
Direct tel. +44 (0) 1372 381857
=================================================================

_______________________________________________
Cplusplus-sig mailing list
Cplusplus-sig at python.org
http://mail.python.org/mailman/listinfo/cplusplus-sig

From binbinsh at gmail.com  Tue Sep  7 17:29:34 2010
From: binbinsh at gmail.com (Binbin Shen)
Date: Tue, 7 Sep 2010 23:29:34 +0800
Subject: [C++-sig] Some questions when making a C++ project as python
	externsions
In-Reply-To: <4C85F236.2040804@gmail.com>
References: <AANLkTim4k+w1S9qijUaw1V0sYRiXo6nU8gsE4vNd=2-w@mail.gmail.com>
	<4C85F236.2040804@gmail.com>
Message-ID: <AANLkTimP8arSMPkWNEx3HbN-AojHuTGHykfaH5QNu9rw@mail.gmail.com>

Thank you very much for your quick reply, I will have a try first. :-)

2010/9/7 Jim Bosch <talljimbo at gmail.com>:
> On 09/07/2010 12:19 AM, Binbin Shen wrote:
>>
>> Hi,
>> I am new to boost.python, and want to provide python interface of a
>> large lab application written in C++. I have written some wrappers and
>> make some classes works in python, but also encounter some problems
>> that cannot be solved by Googling, and I do not want to modify the
>> source of the C++ application, some of the problems are as follows:
>>
>
> Here's an obvious-only-once-you've-seen-it trick that applies to a lot of
> your questions: ?if a C++ function isn't in a form where you can wrap it
> with Boost.Python, write a new C++ function that calls it and wrap that.
> ?This even works for member functions; if the first argument of a free
> function is a class type, it can be wrapped in Python as a member function
> of that class.
>
>
>> 1. Some C++ functions have return the result by parameters like this:
>> void add(std::wstring& ?a, const wstring b) {
>> ? ? a+=b;
>> }
>>>>>
>>>>> a = unicode()
>>>>> b = u"test"
>>>>> add(a, b) ? ? ? ? ? ? ? # error when compiling, unicode can not convert
>>>>> to wstring automatically
>>
>> I should pass a python unicode into this function and return by "a"
>> implicitly.
>>
>
> Unicode may be a little tricky; I don't think Boost.Python handles it at all
> automatically. ?You may have to use the Python C-API a bit (others on this
> list may know more). ?You'll probably want to make one argument of your C++
> wrapper function a boost::python::object; you can then extract a PyObject*
> with the .ptr() member function (boost::python::object is basically a fancy
> smart pointer for PyObject*). ?You can then return a boost::python::object
> containing a real Python unicode object.
>
> Without the unicode part, turning a result-by-arg function into a
> result-by-return is of course easy to solve with a one-line C++ wrapper
> function (and one of the wrapper-generators, Py++, can do this automatically
> for you in some cases).
>
>
>> 2. Some C++ functions show pass a "void*" pointer that pointing a
>> pre-allocated memory into the function, like this:
>> void read(void *memory, int size, int count)
>> {
>> ? ? ? // copy something into memory
>> }
>>
>> How can I make a void pointer in python and pass it to read()?
>>
>
> That depends on what you want that void pointer points to. ?Is it a wrapped
> C++ class, something that maps to a Python built-in type (or a numpy array),
> or is it just raw, unallocated memory?
>
> What do you want to do with it in Python? ?Just pass it to another wrapped
> C++ function?
>
> There are solutions for all of these, but they all require some work.
>
>
>> 3. Some virtual function with default arguments, like this:
>> class foo {
>> public:
>> ? ? virtual int add(int x, int y = 10) { return x+y; }
>> };
>> and the wrapper looks like:
>> struct fooWrap {
>> ? ? int add(int x, int y) {
>> ? ? ? ? if (override add_ = this->get_override("add")
>> ? ? ? ? ? ? return add_(x, y);
>> ? ? ? ? return this->foo::add(x, y);
>> ? ? }
>> ? ? int default add(int x, int y) {
>> ? ? ? ? return this->foo::add(x, y);
>> ? ? }
>> }
>> and
>> ? .def("add",&foo::add,&fooWrap::default_add);
>> but when dealing with overloads, it looks like:
>> BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS(add_overloads, add, 1, 2)
>> and
>> ? ?.def("add",&foo::add, add_overloads())
>>
>> How can I write these two together?
>
> Not sure; I just haven't used the OVERLOADS macros much. ?Hopefully someone
> else can answer.
>
>>
>> 4. Some functions have variable length parameter:
>> std::wstring format(const wchar_t* strFormat, ...) {
>> ? ? // use va_list, va_start, va_end...
>> }
>>
>> how boost.python deal with it?
>>
>
> It can't directly. ?You'll have to write a C++ wrapper function. ?You may
> want to look into boost::python::raw_function (see the reference
> documentation), which lets you write a Python function with a variable
> number of arguments.
>
>
>> 5. The last question is that I do not quiet understand the policies,
>> :-(, is there any detail documents introduce them:
>> with_custodian_and_ward: Ties lifetimes of the arguments
>> with_custodian_and_ward_postcall: Ties lifetimes of the arguments and
>> results
>> return_internal_reference: Ties lifetime of one argument to that of result
>> return_value_policy<T> ?with T one of:
>> reference_existing_object: naive (dangerous) approach object
>> copy_const_reference: Boost.Python v1 approach
>> copy_non_const_reference:
>> manage_new_object: Adopt a pointer and hold the instance
>> return_by_value
>> return_by_reference
>> return_opaque_pointer
>>
>> I use boost 1.42 with gcc 4.4.3 in Gentoo Linux.
>>
>> Any advice would be gratefully received.
>>
>
> The ones you'll probably use most often are:
>
> - return_internal_reference: Main use is when a class returns a pointer or
> reference to a data member. ?It keeps the Python object wrapping the class
> instance alive as long as the Python object wrapping the data member is
> alive.
>
> - return_value_policy + copy_const_reference or copy_non_const_reference:
> Instead of wrapping a reference, invoke the copy constructor and return the
> new object.
>
> - manage_new_object: assumes the return value was allocated by the new
> operator and invokes delete when Python is done with it.
>
> When you return by value or smart pointer, there's generally no need to use
> call policies, and Boost.Python is prettying good at throwing up a compile
> error when you need to use them.
>
>
> Hope that helps!
>
> Jim Bosch
>

From junweizhang2006 at gmail.com  Tue Sep  7 22:25:33 2010
From: junweizhang2006 at gmail.com (Junwei Zhang)
Date: Tue, 7 Sep 2010 15:25:33 -0500
Subject: [C++-sig] Question about using IT++ in boost.python
Message-ID: <AANLkTinAyY3Go_Z_0VQW24K9z54bY3OQ2zqnTcQFr8FL@mail.gmail.com>

Hi,

I am now considering using boost.python to integrate some IT++(C++
scientific library)
codes to python.
But when  I do some simple example
to wrapper this func
int vfunc(int n)
{
    vec a = linspace(0,n,10); // already include IT++ and using its namespace
    vec b= linspace(0,2*n,10);//
    std::cout<<a<<'\n'<<b<<'\n';
    return a*b;



}

 def("vfunc", vfunc);

It compiles but when import the module
there is error
raceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: ./hello.so: undefined symbol: _ZN4itpp8linspaceEddi

any suggestion is appreciate!!


-- 
Junwei Zhang
Office Phone:402-472-1968
www.junweizhang.com
Department of Electrical Engineering
University of Nebraska-Lincoln
209N Walter Scott Engineering Center
P.O. Box 880511
Lincoln, NE 68588-0511

From talljimbo at gmail.com  Wed Sep  8 00:48:46 2010
From: talljimbo at gmail.com (Jim Bosch)
Date: Tue, 07 Sep 2010 15:48:46 -0700
Subject: [C++-sig] Question about using IT++ in boost.python
In-Reply-To: <AANLkTinAyY3Go_Z_0VQW24K9z54bY3OQ2zqnTcQFr8FL@mail.gmail.com>
References: <AANLkTinAyY3Go_Z_0VQW24K9z54bY3OQ2zqnTcQFr8FL@mail.gmail.com>
Message-ID: <4C86C14E.3080209@gmail.com>

On 09/07/2010 01:25 PM, Junwei Zhang wrote:
> Hi,
>
> I am now considering using boost.python to integrate some IT++(C++
> scientific library)
> codes to python.
> But when  I do some simple example
> to wrapper this func
> int vfunc(int n)
> {
>      vec a = linspace(0,n,10); // already include IT++ and using its namespace
>      vec b= linspace(0,2*n,10);//
>      std::cout<<a<<'\n'<<b<<'\n';
>      return a*b;
>
>
>
> }
>
>   def("vfunc", vfunc);
>
> It compiles but when import the module
> there is error
> raceback (most recent call last):
>    File "<stdin>", line 1, in<module>
> ImportError: ./hello.so: undefined symbol: _ZN4itpp8linspaceEddi
>
> any suggestion is appreciate!!
>
>

It looks like you need to link your Python module (it's a shared 
library) against the IT++ shared library.

Jim Bosch



From junweizhang2006 at gmail.com  Wed Sep  8 21:22:27 2010
From: junweizhang2006 at gmail.com (Junwei Zhang)
Date: Wed, 8 Sep 2010 14:22:27 -0500
Subject: [C++-sig] How to expose operator[] in boost.python
Message-ID: <AANLkTi=GJHJS1_VzhN6bJLP8nhnL=FS5tr+J-8EUrE=g@mail.gmail.com>

Hi,

If I have a class support operator[] and operator() for indexing purpose.
How can I wrapper it in python?

-- 
Junwei Zhang
Office Phone:402-472-1968
junweizhang2006 at gmail.com
www.junweizhang.com
Department of Electrical Engineering
University of Nebraska-Lincoln
209N Walter Scott Engineering Center
P.O. Box 880511
Lincoln, NE 68588-0511

From charlessolar at gmail.com  Wed Sep  8 21:30:46 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Wed, 8 Sep 2010 14:30:46 -0500
Subject: [C++-sig] How to expose operator[] in boost.python
In-Reply-To: <AANLkTi=GJHJS1_VzhN6bJLP8nhnL=FS5tr+J-8EUrE=g@mail.gmail.com>
References: <AANLkTi=GJHJS1_VzhN6bJLP8nhnL=FS5tr+J-8EUrE=g@mail.gmail.com>
Message-ID: <AANLkTikh24FNZRR_bu1YJny4YVuF4vWqfu6HQDLewrLu@mail.gmail.com>

Use the python special function __getitem__

So, something like this:

.def( "__getitem__", &Foo::operator[], boost::python::arg( "index" ),
boost::python::return_internal_reference<>() )

If you want your container iteratable, see the docs for boost::python::range.

Charles

On Wed, Sep 8, 2010 at 2:22 PM, Junwei Zhang <junweizhang2006 at gmail.com> wrote:
> Hi,
>
> If I have a class support operator[] and operator() for indexing purpose.
> How can I wrapper it in python?
>
> --
> Junwei Zhang
> Office Phone:402-472-1968
> junweizhang2006 at gmail.com
> www.junweizhang.com
> Department of Electrical Engineering
> University of Nebraska-Lincoln
> 209N Walter Scott Engineering Center
> P.O. Box 880511
> Lincoln, NE 68588-0511
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>

From kubol at kormoran.net  Thu Sep  9 13:50:18 2010
From: kubol at kormoran.net (Jakub Zytka)
Date: Thu, 09 Sep 2010 13:50:18 +0200
Subject: [C++-sig] pyplusplus bug related to boost::equality_comparable (and
 possibly other operators as well)
Message-ID: <4C88C9FA.80301@kormoran.net>

Actually, I haven't had time to investigate it deeper yet, but it seems to be a 
pyplusplus's bug. I also haven't found anything relevant on the net, so here we go:

Test case:
class to be exposed (file a.h)

#include <boost/operators.hpp>

class A: boost::equality_comparable<A>
{
public:
         A(int a): a(a) {};

         bool operator==(A const & other) const
         {
                 return this->a == other.a;
         }
private:
         int a;
};

generator used (gen.py):
from pyplusplus import module_builder

mb = module_builder.module_builder_t(files = ['a.h'])

mb.decl('alloca').exclude()

mb.build_code_creator( module_name = 'testA')
mb.write_module('testA.cpp')

file generated:
// This file has been generated by Py++.

#include "boost/python.hpp"

#include "a.h"

namespace bp = boost::python;

BOOST_PYTHON_MODULE(testA){
     { //::A
         typedef bp::class_< A > A_exposer_t;
         A_exposer_t A_exposer = A_exposer_t( "A", bp::init< int >(( 
bp::arg("a") )) );
         bp::scope A_scope( A_exposer );
         bp::implicitly_convertible< int, A >();
         A_exposer.def( bp::self == bp::self );
	// OOPS! something is missing here
     }
}


C++ class test:
#include <iostream>
#include "a.h"

using namespace std;

int main()
{
         A a1(3);
         A a2(3);
         cout << "a1 == a2: " << (a1 == a2) << endl;
         cout << "a1 != a2: " << (a1 != a2) << endl;
         return 0;
}

C++ class test result:
a1 == a2: 1
a1 != a2: 0

python test:
 >>> import testA
 >>> a1 = testA.A(3)
 >>> a2 = testA.A(3)
 >>> a1 == a2
True
 >>> a1 != a2
True # drat!


The reason: missing A_exposer.def( bp::self != bp::self ); in the generated code;


If any of you think it is easy to fix in pyplusplus (Roman, perhaps you're the 
one to know best?) I may try. I'm also considering a workaround of checking 
bases for all the classes and add missing "self != self" for all those which 
derive from equality_comparable;
Perhaps there is the same problem with less_than_comparable and others.


regards,
Kuba

From roman.yakovenko at gmail.com  Thu Sep  9 20:08:43 2010
From: roman.yakovenko at gmail.com (Roman Yakovenko)
Date: Thu, 9 Sep 2010 21:08:43 +0300
Subject: [C++-sig] pyplusplus bug related to boost::equality_comparable
 (and possibly other operators as well)
In-Reply-To: <4C88C9FA.80301@kormoran.net>
References: <4C88C9FA.80301@kormoran.net>
Message-ID: <AANLkTinU7y75TNEZJT-kr7JSDwEz1Vj5yGkC-eeLPAH7@mail.gmail.com>

On Thu, Sep 9, 2010 at 2:50 PM, Jakub Zytka <kubol at kormoran.net> wrote:
> Actually, I haven't had time to investigate it deeper yet, but it seems to
> be a pyplusplus's bug. I also haven't found anything relevant on the net, so
> here we go:
>
> Test case:
> class to be exposed (file a.h)
>
> #include <boost/operators.hpp>
>
> class A: boost::equality_comparable<A>
> {
> public:
> ? ? ? ?A(int a): a(a) {};
>
> ? ? ? ?bool operator==(A const & other) const
> ? ? ? ?{
> ? ? ? ? ? ? ? ?return this->a == other.a;
> ? ? ? ?}
> private:
> ? ? ? ?int a;
> };
>
> generator used (gen.py):
> from pyplusplus import module_builder
>
> mb = module_builder.module_builder_t(files = ['a.h'])
>
> mb.decl('alloca').exclude()
>
> mb.build_code_creator( module_name = 'testA')
> mb.write_module('testA.cpp')
>
> file generated:
> // This file has been generated by Py++.
>
> #include "boost/python.hpp"
>
> #include "a.h"
>
> namespace bp = boost::python;
>
> BOOST_PYTHON_MODULE(testA){
> ? ?{ //::A
> ? ? ? ?typedef bp::class_< A > A_exposer_t;
> ? ? ? ?A_exposer_t A_exposer = A_exposer_t( "A", bp::init< int >((
> bp::arg("a") )) );
> ? ? ? ?bp::scope A_scope( A_exposer );
> ? ? ? ?bp::implicitly_convertible< int, A >();
> ? ? ? ?A_exposer.def( bp::self == bp::self );
> ? ? ? ?// OOPS! something is missing here
> ? ?}
> }
>
>
> C++ class test:
> #include <iostream>
> #include "a.h"
>
> using namespace std;
>
> int main()
> {
> ? ? ? ?A a1(3);
> ? ? ? ?A a2(3);
> ? ? ? ?cout << "a1 == a2: " << (a1 == a2) << endl;
> ? ? ? ?cout << "a1 != a2: " << (a1 != a2) << endl;
> ? ? ? ?return 0;
> }
>
> C++ class test result:
> a1 == a2: 1
> a1 != a2: 0
>
> python test:
>>>> import testA
>>>> a1 = testA.A(3)
>>>> a2 = testA.A(3)
>>>> a1 == a2
> True
>>>> a1 != a2
> True # drat!
>
>
> The reason: missing A_exposer.def( bp::self != bp::self ); in the generated
> code;
>
>
> If any of you think it is easy to fix in pyplusplus (Roman, perhaps you're
> the one to know best?) I may try.

I don't think fixing py++ is the right approach.

> I'm also considering a workaround of
> checking bases for all the classes and add missing "self != self" for all
> those which derive from equality_comparable;
> Perhaps there is the same problem with less_than_comparable and others.

I would implement the following solution:
1. Introduce new "def_visitor" (
http://www.boost.org/doc/libs/1_44_0/libs/python/doc/v2/def_visitor.html
) - so it will register all member function from the appropriate class
   In our case, equality_comparable, the visitor will register == and !=
  ( this code could be contributed later to the Boost.Python )

2. Apply that visitors on all relevant classes using py++

Does it help?

-- 
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/

From kubol at kormoran.net  Fri Sep 10 10:28:34 2010
From: kubol at kormoran.net (Jakub Zytka)
Date: Fri, 10 Sep 2010 10:28:34 +0200
Subject: [C++-sig] pyplusplus bug related to boost::equality_comparable
 (and possibly other operators as well)
In-Reply-To: <AANLkTinU7y75TNEZJT-kr7JSDwEz1Vj5yGkC-eeLPAH7@mail.gmail.com>
References: <4C88C9FA.80301@kormoran.net>
	<AANLkTinU7y75TNEZJT-kr7JSDwEz1Vj5yGkC-eeLPAH7@mail.gmail.com>
Message-ID: <4C89EC32.2030001@kormoran.net>

On 09/09/10 20:08, Roman Yakovenko wrote:
> I don't think fixing py++ is the right approach.

>> I'm also considering a workaround of
>> checking bases for all the classes and add missing "self != self" for all
>> those which derive from equality_comparable;
>> Perhaps there is the same problem with less_than_comparable and others.
>
> I would implement the following solution:
> 1. Introduce new "def_visitor" (
> http://www.boost.org/doc/libs/1_44_0/libs/python/doc/v2/def_visitor.html
> ) - so it will register all member function from the appropriate class
>     In our case, equality_comparable, the visitor will register == and !=
>    ( this code could be contributed later to the Boost.Python )
>
> 2. Apply that visitors on all relevant classes using py++
>
> Does it help?
Partially ;)
I don't understand the concept with a new visitor. Creating a visitor for eg. 
equality_comparable doesn't solve the problem in general (and btw, 
equality_comparable doesn't provide member functions)
I think all the needed visitors are already there (eg. boost::python::self != 
boost::python::self). The problem is that pyplusplus doesn't use them (or - to 
be specific - it *sometimes* doesn't use them).

Why I consider it as a py++ bug? The problem is *sometimes* py++ adds some 
visitors to a class, and sometimes not. I suspect (not 100% verified) it doesn't 
work if an operator is declared as a non member function. I haven't found such 
warning in the docs, thus my surprise. If it is there, well, my fault. It 
escaped me.
The uncertainty about operator== and operator!= is very unfortunate, because in 
python you have == or != by default, and in python code you don't know whether 
you're using pythonic comparison or the one you've defined in C++.
So perhaps it is at least worth pointing out in docs (if it is not there, and 
didn't escape ;).
I'd also add a tip that for each class_ one may (or should) define __cmp__ to 
raise some fancy exception in order to avoid problems as below:
b1 = BoundClass(...)
b2 = BoundClass(...)
b1 == b2 # are we using pythonic pointer comparison or C++ defined one?

If __cmp__ is defined to raise exception:
1. if operator== is defined in C++ and correct visitor is applied then python's 
__eq__ will be used, and comparison works as expected (that is: as in C++)
2. if operator== is not defined or no visitor has been applied (__eq__ is not 
defined) __cmp__ will be called and exception will happen. There will be no need 
to trace bugs that occurred in seemingly impossible branches.

In my case I applied point 2 as a workaround: I look for all the operators 
(member or not) and apply related visitors on the classes that are parameters of 
these operators. This is far for ideal, but works in most cases.
Having exception raising __cmp__ I'm able to spot problems immediately.

regards,
Kuba

From mbiasini at bluewin.ch  Fri Sep 10 11:44:22 2010
From: mbiasini at bluewin.ch (mbiasini at bluewin.ch)
Date: Fri, 10 Sep 2010 09:44:22 +0000 (GMT+00:00)
Subject: [C++-sig] __module__ of boost.python functions?
Message-ID: <12434767.54831284111862760.JavaMail.webmail@ps30zhh.bluewin.ch>

Hi everyone,

I'm trying to overload the python help function and hook it up to an external documentation system. I'm 
using the  qualified name of the object to perform a lookup in a documentation database. So far I've got it working for 
almost everything defined in Python (classes, functions, etc.), but have difficulties doing the same for boost python 
objects. If I do

MyCFunc.__module__

I get: AttributeError: 'Boost.Python.function' object has no attribute 
'__module__', 

even though I can see that __module__ is defined when I do a dir(MyCFunc). Is there a technical reason 
why functions and properties exported with boost python don't have the __module__ property defined? What is the 
recommended way of getting the qualified name for those? 

Thanks you very much for any hints! 

Best,
Marco

From junweizhang2006 at gmail.com  Tue Sep 14 07:08:44 2010
From: junweizhang2006 at gmail.com (Junwei Zhang)
Date: Tue, 14 Sep 2010 00:08:44 -0500
Subject: [C++-sig] Question about creation package
Message-ID: <AANLkTin6AYSGjvFd2zWNUa8y1nzQWejn5WykC4ncq0Eh@mail.gmail.com>

Hi,

I follow the boost.python tutorial
but it does not work out

I create the directory structure as

itppl/
    __init__py
    itpp.so

and add it to  use sys.path.append
but
in python

import itppl.itpp

error  No module named itppl.tpp

What 's wrong with this?

-- 
Junwei Zhang
Office Phone:402-472-1968
junweizhang2006 at gmail.com
www.junweizhang.com
Department of Electrical Engineering
University of Nebraska-Lincoln
209N Walter Scott Engineering Center
P.O. Box 880511
Lincoln, NE 68588-0511

From paniq at paniq.org  Tue Sep 14 21:12:39 2010
From: paniq at paniq.org (Leonard Ritter)
Date: Tue, 14 Sep 2010 21:12:39 +0200
Subject: [C++-sig] gtkglext & mesa = __cxa_allocate_exception
Message-ID: <AANLkTimhynH0F=JYLx_apAMKbENOyniaGQsp3rLGkBzq@mail.gmail.com>

as a quick follow-up to this two year old post here:

http://www.mail-archive.com/cplusplus-sig at python.org/msg00245.html

even with linking to libstdc++ first, as i always do, i'm hitting upon
this error on ubuntu 10.04 yet again, after i switched the nvidia
driver to nouveau.

it seems that when gtk.glext is imported in python, and a boost python
module hits a StopIteration exception (as it is usual for generators),
the exception is not caught and the app segfaults.

so apparently, something within gtkglext seems to set-up the problem -
unfortunately, the issue is not traceable. :/

From paniq at paniq.org  Tue Sep 14 21:18:04 2010
From: paniq at paniq.org (Leonard Ritter)
Date: Tue, 14 Sep 2010 21:18:04 +0200
Subject: [C++-sig] gtkglext & mesa = __cxa_allocate_exception
In-Reply-To: <AANLkTimhynH0F=JYLx_apAMKbENOyniaGQsp3rLGkBzq@mail.gmail.com>
References: <AANLkTimhynH0F=JYLx_apAMKbENOyniaGQsp3rLGkBzq@mail.gmail.com>
Message-ID: <AANLkTinbsDupUsw=aqNAfvzumC9eDnKWRfGyONYFbD7A@mail.gmail.com>

On Tue, Sep 14, 2010 at 9:12 PM, Leonard Ritter <paniq at paniq.org> wrote:
> it seems that when gtk.glext is imported in python, and a boost python
> module hits a StopIteration exception (as it is usual for generators),
> the exception is not caught and the app segfaults.

that is, unless OpenGL.GL is imported before gtk.glext - in this case,
no segfaults; it's the translation of a sites suggestion to use "env
LD_PRELOAD=libGL.so python script.py", which is essentially the same
thing.

i wonder what kind of voodoo is at work here :)

From kubol at kormoran.net  Wed Sep 15 09:33:11 2010
From: kubol at kormoran.net (Jakub Zytka)
Date: Wed, 15 Sep 2010 09:33:11 +0200
Subject: [C++-sig] gtkglext & mesa = __cxa_allocate_exception
In-Reply-To: <AANLkTinbsDupUsw=aqNAfvzumC9eDnKWRfGyONYFbD7A@mail.gmail.com>
References: <AANLkTimhynH0F=JYLx_apAMKbENOyniaGQsp3rLGkBzq@mail.gmail.com>
	<AANLkTinbsDupUsw=aqNAfvzumC9eDnKWRfGyONYFbD7A@mail.gmail.com>
Message-ID: <4C9076B7.80208@kormoran.net>

On 09/14/10 21:18, Leonard Ritter wrote:
> On Tue, Sep 14, 2010 at 9:12 PM, Leonard Ritter<paniq at paniq.org>  wrote:
>> it seems that when gtk.glext is imported in python, and a boost python
>> module hits a StopIteration exception (as it is usual for generators),
>> the exception is not caught and the app segfaults.
>
> that is, unless OpenGL.GL is imported before gtk.glext - in this case,
> no segfaults; it's the translation of a sites suggestion to use "env
> LD_PRELOAD=libGL.so python script.py", which is essentially the same
> thing.
>
> i wonder what kind of voodoo is at work here :)
Last time I run into this kind of problem it was because python was using 
different memory allocator than the library I was importing (that is: the 
library was linked against our slightly modified tcmalloc)

From David.Aldrich at EU.NEC.COM  Wed Sep 15 11:37:00 2010
From: David.Aldrich at EU.NEC.COM (David Aldrich)
Date: Wed, 15 Sep 2010 10:37:00 +0100
Subject: [C++-sig] Advice sought on making a large C++ application
	a	Python	extension
In-Reply-To: <27AD07BD25612C40AF5C78CB49D09B4967DEFF97F9@34093-MBX-C08.mex07a.mlsrvr.com>
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
	<27AD07BD25612C40AF5C78CB49D09B4967DEFF97F9@34093-MBX-C08.mex07a.mlsrvr.com>
Message-ID: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D220A@EUEXCLU01.EU.NEC.COM>

Hi Ben

Thanks very much for your reply. Sorry that my response has been slow but I have been trying to digest yours. I have also taken a detour to investigate CMake.

>As to how to make your functionality available with a Python 
>extension - I'm not sure exactly what kind of app you have, 
>but is it possible to make the Python portion only an 
>interface to the underlying C++ code? That's basically what 
>we did with our application - write a wrapper layer on top 
>of the ordinary C++ API that implemented a Python API. This 
>also allowed us to build the C++ application, and then with
>an additional build step (in GNU make) we were able to 
>compile the python interface layer.

I'm sorry to say that I don't really understand the concept of making the Python portion only an interface to the underlying C++ code. Please could you explain what that means? 

Best regards

David

From ndbecker2 at gmail.com  Wed Sep 15 18:01:34 2010
From: ndbecker2 at gmail.com (Neal Becker)
Date: Wed, 15 Sep 2010 12:01:34 -0400
Subject: [C++-sig] Advice sought on making a large C++ application a
	Python	extension
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
	<27AD07BD25612C40AF5C78CB49D09B4967DEFF97F9@34093-MBX-C08.mex07a.mlsrvr.com>
Message-ID: <i6qqkv$gch$1@dough.gmane.org>

Does pybindgen interoperate with numpy?  That would be required for me to 
use it.

Ben Fitzpatrick wrote:

> David,
> 
> I'm still a newbie on this list myself, but I will tell you that for the
> larger project which we were attempting to wrap, we had excellent luck
> with Gustavo Carniero's Pybindgen project. We found it to be much simpler
> to work with than boost.python and py++, and also it compiled much faster.
> 
> As to how to make your functionality available with a Python extension -
> I'm not sure exactly what kind of app you have, but is it possible to make
> the Python portion only an interface to the underlying C++ code? That's
> basically what we did with our application - write a wrapper layer on top
> of the ordinary C++ API that implemented a Python API. This also allowed
> us to build the C++ application, and then with an additional build step
> (in GNU make) we were able to compile the python interface layer.
> 
> It may be that you need to make all of your classes into python
> extensions, in which case py++ and boost.python may be the better tool, I
> am not certain, but for writing just an interface layer, our experience
> was that Pybindgen was easier.
> 
> Hope this helps!
> 
> -Ben
> 
> -----Original Message-----
> From: cplusplus-sig-bounces+bfitzpatrick=vtiinstruments.com at python.org
> [mailto:cplusplus-sig-bounces+bfitzpatrick=vtiinstruments.com at python.org]
> On Behalf Of David Aldrich Sent: Monday, September 06, 2010 11:16 AM To:
> Development of Python/C++ integration Subject: [C++-sig] Advice sought on
> making a large C++ application a Python extension
> 
> Hi
> 
> We have a large C++ application that we develop in-house. It consists of a
> large number of source files, some of which are linked directly and some
> of which are first built as static libraries.
> 
> It has been suggested that we make the application's functionality
> available as a Python extension.  Simplistically speaking, we could then
> let users replace our C++ main() by a Python script.
> 
> I have only looked at the boost.python tutorial that demonstrates a 'Hello
> world' Python extension.  For our project we would be dealing with
> something very much bigger, with many classes, singleton classes and
> (possibly) global data.  My question is, would it be practical to make all
> of those entities Python extensions?
> 
> We would also want to continue to support the application as a wholly C++
> application.   Currently, we use gnu make to build it.  Would I have to
> add a parallel build system using bjam to build the extensions? And then
> maintain both make's makefiles and bjam's Jamroot files?
> 
> Can .pyd files be loaded by the C++ linker?
> 
> Any advice would be gratefully received.
> 
> Best regards
> 
> David
> 
> =================================================================
> David Aldrich, NEC Telecom MODUS, Ltd,
> Cleeve Road, Leatherhead, Surrey, KT22 7SA, UK
> Direct tel. +44 (0) 1372 381857
> =================================================================
> 
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig



From bfitzpatrick at vtiinstruments.com  Wed Sep 15 19:39:04 2010
From: bfitzpatrick at vtiinstruments.com (Ben Fitzpatrick)
Date: Wed, 15 Sep 2010 12:39:04 -0500
Subject: [C++-sig] Advice sought on making a large C++
	application	a	Python	extension
In-Reply-To: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D220A@EUEXCLU01.EU.NEC.COM>
References: <0CCCA5445D7FCB4298BBF0739D4FDD520260A85D1DFB@EUEXCLU01.EU.NEC.COM>
	<27AD07BD25612C40AF5C78CB49D09B4967DEFF97F9@34093-MBX-C08.mex07a.mlsrvr.com>
	<0CCCA5445D7FCB4298BBF0739D4FDD520260A85D220A@EUEXCLU01.EU.NEC.COM>
Message-ID: <27AD07BD25612C40AF5C78CB49D09B4967DF0E4BCF@34093-MBX-C08.mex07a.mlsrvr.com>


>I'm sorry to say that I don't really understand the concept of making the Python portion only an interface to the >underlying C++ code. Please could you explain what that means? 

David,

Sorry, I must not have communicated this well. I'll try again. It depends on how your code works, but the idea is, we generate a shared library with some (large) number of entry points. This is our API, basically. We allow users to program against our API, but there's a ton of other code within the shared library that they don't see, or don't get access to.

So, we still generate the C++ shared library, but in addition we generate a layer using Pybindgen that sits on top of the API and exposes it via Python. We didn't have to wrap all the internal function calls in the shared library in Python - that would have been too much work. Instead we built a wrapper around our shared library that implements all the same calls our shared library's API implements, and when they are accessed just makes a call via pybindgen into the existing C++ shared library. Inside the library, we don't care what happens - C++ functions can call other C++ functions, etc - and eventually a result comes back to the python wrapper layer, is translated from C++, and is returned to Python.

For something that's more of a program instead of a library, I imagine you could do something similar - instead of wrapping all of your modules or functions, replace the main() function of your program with a python version - and only bother generating a wrapper for the functions called directly from main. I haven't tried the main-replacement bit, but I can verify that for the shared library, it works just fine.

-Ben

From rwgk at yahoo.com  Fri Sep 17 04:02:30 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Thu, 16 Sep 2010 19:02:30 -0700 (PDT)
Subject: [C++-sig] __module__ of boost.python functions?
In-Reply-To: <12434767.54831284111862760.JavaMail.webmail@ps30zhh.bluewin.ch>
References: <12434767.54831284111862760.JavaMail.webmail@ps30zhh.bluewin.ch>
Message-ID: <954541.26428.qm@web111408.mail.gq1.yahoo.com>

Hi Marco,
Could you try if the attached patch does the trick for you?
(I've tested it with Python 2.7.)
Ralf



----- Original Message ----
From: "mbiasini at bluewin.ch" <mbiasini at bluewin.ch>
To: cplusplus-sig at python.org
Sent: Fri, September 10, 2010 2:44:22 AM
Subject: [C++-sig] __module__ of boost.python functions?

Hi everyone,

I'm trying to overload the python help function and hook it up to an external 
documentation system. I'm 

using the  qualified name of the object to perform a lookup in a documentation 
database. So far I've got it working for 

almost everything defined in Python (classes, functions, etc.), but have 
difficulties doing the same for boost python 

objects. If I do

MyCFunc.__module__

I get: AttributeError: 'Boost.Python.function' object has no attribute 
'__module__', 

even though I can see that __module__ is defined when I do a dir(MyCFunc). Is 
there a technical reason 

why functions and properties exported with boost python don't have the 
__module__ property defined? What is the 

recommended way of getting the qualified name for those? 

Thanks you very much for any hints! 

Best,
Marco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bp_func_module_patch
Type: application/octet-stream
Size: 1867 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100916/a46acacc/attachment.obj>

From simwarg at gmail.com  Tue Sep 21 15:53:40 2010
From: simwarg at gmail.com (Simon Warg)
Date: Tue, 21 Sep 2010 15:53:40 +0200
Subject: [C++-sig] Get parameter info of function or method
References: <AANLkTi=ie1Y5kZ1+nrS-s7an_H7mfNB5X8pkt7+aaKkH@mail.gmail.com>
Message-ID: <232D5272-9CE2-4B54-90EB-37F265EE4DBF@gmail.com>

>
> Hi,
I'm currently generating doc for my live boost.python classes,  
instances and function. But I can't find any way to retrieve the  
parameter info of functions and its signature because it has no  
__code__ attribute due to it's written I C++. Any solution to this?

From rwgk at yahoo.com  Tue Sep 21 18:15:18 2010
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Tue, 21 Sep 2010 09:15:18 -0700 (PDT)
Subject: [C++-sig] Get parameter info of function or method
In-Reply-To: <232D5272-9CE2-4B54-90EB-37F265EE4DBF@gmail.com>
References: <AANLkTi=ie1Y5kZ1+nrS-s7an_H7mfNB5X8pkt7+aaKkH@mail.gmail.com>
	<232D5272-9CE2-4B54-90EB-37F265EE4DBF@gmail.com>
Message-ID: <566901.81107.qm@web111402.mail.gq1.yahoo.com>

Boost.Python generates docstrings during compilation, which you can retrieve 
with
obj.__doc__
Some examples:
http://www.boost.org/doc/libs/1_44_0/libs/python/doc/v2/docstring_options.html
Ralf



----- Original Message ----
From: Simon Warg <simwarg at gmail.com>
To: "cplusplus-sig at python.org" <cplusplus-sig at python.org>
Sent: Tue, September 21, 2010 6:53:40 AM
Subject: [C++-sig] Get parameter info of function or method

> 
> Hi,
I'm currently generating doc for my live boost.python classes, instances and 
function. But I can't find any way to retrieve the parameter info of functions 
and its signature because it has no __code__ attribute due to it's written I 
C++. Any solution to this?

From charlessolar at gmail.com  Wed Sep 22 23:29:02 2010
From: charlessolar at gmail.com (Charles Solar)
Date: Wed, 22 Sep 2010 16:29:02 -0500
Subject: [C++-sig] bp::wrapper class returning reference type
Message-ID: <AANLkTinjGZr6xh3im-FTymAvbWGFbcRAtrta2naTwq_R@mail.gmail.com>

I have a bp::wrapper wrapping an interface for use in python.  Several
functions return reference types of other objects, for example:

class IFoo
{
    virtual const IBar& getBar() const = 0;
}

I found that to get boost python to handle this scenario correctly and
eliminate any 'returning address of local variable' warnings I had to
trick the result converter like so:

class Foo_wrapper : IFoo, bp::wrapper< IFoo >
{
    Bar _bar;
    const IBar& getBar() const
    {
        _bar = *(IBar*)this->get_override( "getBar" )();
        // return *(IBar*)this->get_override( "getBar" )();   <- works
as well but maybe less safe?
    }
}

I was messing with this problem for the last day and a half and this
finally worked.  From what I understand the method converter is
tricked into returning an IBar* by the cast, then I use the copy
constructor to make a concrete bar locally which is finally returned
as a reference to the calling code.

My first concern is if this is a 'proper' solution for this problem.
I found that by just doing
const IBar& getBar() const
{
    return (IBar&)this->get_override( "getBar" )();
}

I would get crashes because the object returned from get_override
would be destroyed before I can copy it.  IBar is abstract so most
'correct' uses will not work.

If this solution is acceptable my question is how is the result
converter dealing with the IBar* cast.  Is it depending on me to
delete the pointer?  Does this solution cause massive memory leaks?
Or will cleaning up the python object clean up the pointer as well?  I
want to implement a 'proper' solution to minimize future potential
problems.

Thank you

From simwarg at gmail.com  Fri Sep 24 16:08:50 2010
From: simwarg at gmail.com (Simon W)
Date: Fri, 24 Sep 2010 16:08:50 +0200
Subject: [C++-sig] operator in return false while operator == return true
Message-ID: <AANLkTimY2OWMVG6eepVAuTd0uqU=a36UECGoM5VmrQs9@mail.gmail.com>

Hi,

Im really scared because I fear a fundamental issue concerning the script
system in my game engine.


In C++, I have a class. In my class I map data to GameObject like:

*class
{
    map<GameObject*, Data> .....
}*

As you see, I use the pointer as key.

When I from c++, in the same class, call a python function I pass the key
like this:

*class::callPythonFunctions()
{

boost::python::get_override("callbackFunction")(boost::python::object(boost::python::ptr(gameobject)));
// the variable gameobj is of type GameObject*
}
*
When I recieve the call in python I do some checks like this:

*def callbackFunction(self, gameobj):
   for x in self.mydict.keys():
     print("Checking")
     print(gameobj == self.mydict)
     print(gameobj in self.mydict)*


The above will print something like:
*....
Checking
True
False
...*

I do have a overloaded == operator. But how can I fix so that python checks
for my pointer and not the PyObject* pointer ?

// Simon
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100924/ee590388/attachment.html>

From seefeld at sympatico.ca  Fri Sep 24 16:27:51 2010
From: seefeld at sympatico.ca (Stefan Seefeld)
Date: Fri, 24 Sep 2010 10:27:51 -0400
Subject: [C++-sig] operator in return false while operator == return true
In-Reply-To: <AANLkTimY2OWMVG6eepVAuTd0uqU=a36UECGoM5VmrQs9@mail.gmail.com>
References: <AANLkTimY2OWMVG6eepVAuTd0uqU=a36UECGoM5VmrQs9@mail.gmail.com>
Message-ID: <4C9CB567.307@sympatico.ca>

  Simon,

I don't quite understand what you are trying to do. Please provide a 
little more detail. Your current mail requires far too much 
second-guessing to be useful.

On 09/24/2010 10:08 AM, Simon W wrote:
> Hi,
>
> Im really scared because I fear a fundamental issue concerning the 
> script system in my game engine.
>
>
> In C++, I have a class. In my class I map data to GameObject like:
>
> *class
> {
>     map<GameObject*, Data> .....
> }*

OK. (Naming this class would help the discussion, though.)

>
> As you see, I use the pointer as key.
>
> When I from c++, in the same class, call a python function I pass the 
> key like this:
>
> *class::callPythonFunctions()
> {
>    
> boost::python::get_override("callbackFunction")(boost::python::object(boost::python::ptr(gameobject))); 
> // the variable /gameobj /is of type GameObject*
> }
> *

OK.

>
> When I recieve the call in python I do some checks like this:
>
> *def callbackFunction(self, gameobj):
>    for x in self.mydict.keys():
>      print("Checking")
>      print(gameobj == self.mydict)
>      print(gameobj in self.mydict)*

This looks wrong. You iterate over 'x', but don't use it in the loop. 
May I assume that 'mydict' relates to the above map<GameObject*, Data> 
in the unnamed class ?

>
>
> The above will print something like:
> /....
> Checking
> True
> False
> .../

This suggests that 'gameobj' compares equal to the 'mydict' object, but 
that it is not itself included in the sequence returned by mydict.keys().


>
> I do have a overloaded == operator. But how can I fix so that python 
> checks for my pointer and not the PyObject* pointer ?

What type do you consider providing an operator== for ?

     Stefan


-- 

       ...ich hab' noch einen Koffer in Berlin...


From simwarg at gmail.com  Fri Sep 24 23:42:35 2010
From: simwarg at gmail.com (Simon W)
Date: Fri, 24 Sep 2010 23:42:35 +0200
Subject: [C++-sig] operator in return false while operator == return true
In-Reply-To: <4C9CB567.307@sympatico.ca>
References: <AANLkTimY2OWMVG6eepVAuTd0uqU=a36UECGoM5VmrQs9@mail.gmail.com>
	<4C9CB567.307@sympatico.ca>
Message-ID: <AANLkTi=uDsP983tHxFTg3TXiRbkT+FRykM3v9d_giJss@mail.gmail.com>

Hey, sorry for the lacking information.

May I declare it as this instead:

C++ Class
class GameObjectManager
{
    map<GameObject*, Data> a_map; // this map has "nothing" to do with
python dict
   GameObject* createGameObject(id)
   {
      Gameobject* obj = new Gameobject(id);
      a_map[ obj ] = ...some data...
      return obj;
    }


   callPython()
  {
      // get game object from map
      obj = a_map.getgameobject()

boost::python::get_override("callbackFunction")(boost::python::object(boost::python::ptr(obj)));
  }

}; // end of class

The python class:


class myRandomClass:
   def __init(self, the_c++_GameobjectManager_pointer):
     self.manager = the_c++_GameobjectManager_pointer
     gameobject = self.manager.createGameObject(1)
#self.manager.createGameObject() returns a pointer to the gameobject created
in c++
     self.mylist[gameobject] = ..some data..

   def callPython(self, gameobj): # at some point, c++ will call this
function and pass the game object we created above with id 1
    for x in self.mylist.keys():
      print(x == gameobj)         # True
      print(gameobj in mylist)    # False

So basically the c++ class and python class have their own dict but I want
them to have the *same *address value in their keys .. is it possible??

It seems, in c++ the key is the address of the game object and in Python it
some other address (but they basically point to the same object/memory). Is
there any way around this *without* changing the type of the key being used,
i.e. keep the key as a pointer.




On Fri, Sep 24, 2010 at 4:27 PM, Stefan Seefeld <seefeld at sympatico.ca>wrote:

>  Simon,
>
> I don't quite understand what you are trying to do. Please provide a little
> more detail. Your current mail requires far too much second-guessing to be
> useful.
>
>
> On 09/24/2010 10:08 AM, Simon W wrote:
>
>> Hi,
>>
>> Im really scared because I fear a fundamental issue concerning the script
>> system in my game engine.
>>
>>
>> In C++, I have a class. In my class I map data to GameObject like:
>>
>> *class
>> {
>>    map<GameObject*, Data> .....
>> }*
>>
>
> OK. (Naming this class would help the discussion, though.)
>
>
>> As you see, I use the pointer as key.
>>
>> When I from c++, in the same class, call a python function I pass the key
>> like this:
>>
>> *class::callPythonFunctions()
>> {
>>
>> boost::python::get_override("callbackFunction")(boost::python::object(boost::python::ptr(gameobject)));
>> // the variable /gameobj /is of type GameObject*
>> }
>> *
>>
>
> OK.
>
>
>> When I recieve the call in python I do some checks like this:
>>
>> *def callbackFunction(self, gameobj):
>>   for x in self.mydict.keys():
>>     print("Checking")
>>     print(gameobj == self.mydict)
>>     print(gameobj in self.mydict)*
>>
>
> This looks wrong. You iterate over 'x', but don't use it in the loop. May I
> assume that 'mydict' relates to the above map<GameObject*, Data> in the
> unnamed class ?
>
>
>>
>> The above will print something like:
>> /....
>> Checking
>> True
>> False
>> .../
>>
>
> This suggests that 'gameobj' compares equal to the 'mydict' object, but
> that it is not itself included in the sequence returned by mydict.keys().
>
>
>
>
>> I do have a overloaded == operator. But how can I fix so that python
>> checks for my pointer and not the PyObject* pointer ?
>>
>
> What type do you consider providing an operator== for ?
>
>    Stefan
>
>
> --
>
>      ...ich hab' noch einen Koffer in Berlin...
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100924/103430c8/attachment.html>

From talljimbo at gmail.com  Sat Sep 25 00:34:36 2010
From: talljimbo at gmail.com (Jim Bosch)
Date: Fri, 24 Sep 2010 15:34:36 -0700
Subject: [C++-sig] operator in return false while operator == return true
In-Reply-To: <AANLkTi=uDsP983tHxFTg3TXiRbkT+FRykM3v9d_giJss@mail.gmail.com>
References: <AANLkTimY2OWMVG6eepVAuTd0uqU=a36UECGoM5VmrQs9@mail.gmail.com>	<4C9CB567.307@sympatico.ca>
	<AANLkTi=uDsP983tHxFTg3TXiRbkT+FRykM3v9d_giJss@mail.gmail.com>
Message-ID: <4C9D277C.7010505@gmail.com>

On 09/24/2010 02:42 PM, Simon W wrote:
> Hey, sorry for the lacking information.
>
> May I declare it as this instead:
>
> C++ Class
> class GameObjectManager
> {
>      map<GameObject*, Data> a_map; // this map has "nothing" to do with
> python dict
>     GameObject* createGameObject(id)
>     {
>        Gameobject* obj = new Gameobject(id);
>        a_map[ obj ] = ...some data...
>        return obj;
>      }
>
>
>     callPython()
>    {
>        // get game object from map
>        obj = a_map.getgameobject()
>
> boost::python::get_override("callbackFunction")(boost::python::object(boost::python::ptr(obj)));
>    }
>
> }; // end of class
>
> The python class:
>
>
> class myRandomClass:
>     def __init(self, the_c++_GameobjectManager_pointer):
>       self.manager = the_c++_GameobjectManager_pointer
>       gameobject = self.manager.createGameObject(1)
> #self.manager.createGameObject() returns a pointer to the gameobject
> created in c++
>       self.mylist[gameobject] = ..some data..
>
>     def callPython(self, gameobj): # at some point, c++ will call this
> function and pass the game object we created above with id 1
>      for x in self.mylist.keys():
>        print(x == gameobj)         # True
>        print(gameobj in mylist)    # False
>
> So basically the c++ class and python class have their own dict but I
> want them to have the *same *address value in their keys .. is it possible??
>
> It seems, in c++ the key is the address of the game object and in Python
> it some other address (but they basically point to the same
> object/memory). Is there any way around this _without_ changing the type
> of the key being used, i.e. keep the key as a pointer.
>

You need to define __eq__, __ne__ and __hash__ in Python in such a way 
that they compare the C++ pointers for equality.  You can do this by 
writing free functions that take a GameObject* as their first argument, 
and wrapping them as Python methods with those special names.

That should be all you need to do.

Be aware that defining __hash__ also tells Python that you promise that 
it's return value is an immutable property of the Python object - i.e. 
the same Python object should never be changed to point to a different 
C++ object (this is hard to do accidentally anyhow in Boost.Python).

Jim



>
>
>
> On Fri, Sep 24, 2010 at 4:27 PM, Stefan Seefeld <seefeld at sympatico.ca
> <mailto:seefeld at sympatico.ca>> wrote:
>
>       Simon,
>
>     I don't quite understand what you are trying to do. Please provide a
>     little more detail. Your current mail requires far too much
>     second-guessing to be useful.
>
>
>     On 09/24/2010 10:08 AM, Simon W wrote:
>
>         Hi,
>
>         Im really scared because I fear a fundamental issue concerning
>         the script system in my game engine.
>
>
>         In C++, I have a class. In my class I map data to GameObject like:
>
>         *class
>         {
>             map<GameObject*, Data> .....
>         }*
>
>
>     OK. (Naming this class would help the discussion, though.)
>
>
>         As you see, I use the pointer as key.
>
>         When I from c++, in the same class, call a python function I
>         pass the key like this:
>
>         *class::callPythonFunctions()
>         {
>
>         boost::python::get_override("callbackFunction")(boost::python::object(boost::python::ptr(gameobject)));
>         // the variable /gameobj /is of type GameObject*
>         }
>         *
>
>
>     OK.
>
>
>         When I recieve the call in python I do some checks like this:
>
>         *def callbackFunction(self, gameobj):
>            for x in self.mydict.keys():
>              print("Checking")
>              print(gameobj == self.mydict)
>              print(gameobj in self.mydict)*
>
>
>     This looks wrong. You iterate over 'x', but don't use it in the
>     loop. May I assume that 'mydict' relates to the above
>     map<GameObject*, Data> in the unnamed class ?
>
>
>
>         The above will print something like:
>         /....
>         Checking
>         True
>         False
>         .../
>
>
>     This suggests that 'gameobj' compares equal to the 'mydict' object,
>     but that it is not itself included in the sequence returned by
>     mydict.keys().
>
>
>
>
>         I do have a overloaded == operator. But how can I fix so that
>         python checks for my pointer and not the PyObject* pointer ?
>
>
>     What type do you consider providing an operator== for ?
>
>         Stefan
>
>
>     --
>
>           ...ich hab' noch einen Koffer in Berlin...
>
>     _______________________________________________
>     Cplusplus-sig mailing list
>     Cplusplus-sig at python.org <mailto:Cplusplus-sig at python.org>
>     http://mail.python.org/mailman/listinfo/cplusplus-sig
>
>
>
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig


From simwarg at gmail.com  Tue Sep 28 21:02:04 2010
From: simwarg at gmail.com (Simon W)
Date: Tue, 28 Sep 2010 21:02:04 +0200
Subject: [C++-sig] TypeError: No to_python (by-value) converter found for
 C++ type: class std::basic_ostream<char, struct std::char_traits<char> >
Message-ID: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>

Hi,
Im trying to expose limited parts of std::ostream like this:
        {
            class_<std::ios_base, boost::noncopyable>("iosbase", init<const
std::ios_base&>())
                ;

            class_<std::ios, bases<std::ios_base>,
boost::noncopyable>("ios", init<std::streambuf*>())
                ;

            std::ostream&(std::ostream::*write)(const char*,
std::streamsize) = &std::ostream::write;

            return class_<std::ostream, bases<std::ios>,
boost::noncopyable>("ostream", no_init)
                .def("write", write ,
return_value_policy<copy_non_const_reference>())
                ;

        }


When I call a python function from a C++ wrapper class:

            virtual Uint32    serialise(const GameObject* obj, std::ostream&
file)
            {
                try
                {
                if(boost::python::override f = get_override("serialise"))
                    return f(boost::python::object(boost::python::ptr(obj)),
boost::python::object(boost::python::ptr(&file)));
                else
                    return ComponentSystem::serialise(obj, file);
                }
                catch(...)
                {
                    PyErr_Print();
                }
            }

I get this error:
*TypeError: No to_python (by-value) converter found for C++ type: class
std::basic_ostream<char,struct std::char_traits<char> >*

This is the python function
*    def serialise(self, gameobj, file):
        if(gameobj in self.components):
            file.write("HEY", 3)*

What's the problem here? Is is because the ostream argument is an reference?
Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100928/e19b817b/attachment.html>

From kubol at kormoran.net  Wed Sep 29 11:38:48 2010
From: kubol at kormoran.net (Jakub Zytka)
Date: Wed, 29 Sep 2010 11:38:48 +0200
Subject: [C++-sig] TypeError: No to_python (by-value) converter found
 for C++ type: class std::basic_ostream<char, struct std::char_traits<char> >
In-Reply-To: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>
References: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>
Message-ID: <4CA30928.4010404@kormoran.net>


 > I get this error:
 > *TypeError: No to_python (by-value) converter found for C++ type: class
 > std::basic_ostream<char,struct std::char_traits<char> >*
 >
 > This is the python function
 > *    def serialise(self, gameobj, file):
 >          if(gameobj in self.components):
 >              file.write("HEY", 3)*
 >
 > What's the problem here? Is is because the ostream argument is an reference?
The problem is you are trying to copy a noncopyable object:

>              return class_<std::ostream, bases<std::ios>,
> boost::noncopyable>("ostream", no_init)
ostream is noncopyable

>                  .def("write", write ,
> return_value_policy<copy_non_const_reference>())
but you define write to return copy (of ostream)


From simwarg at gmail.com  Wed Sep 29 14:08:13 2010
From: simwarg at gmail.com (Simon W)
Date: Wed, 29 Sep 2010 14:08:13 +0200
Subject: [C++-sig] Read and write byte streams python<-->c++
Message-ID: <AANLkTi=wJuFD-dW=src7Hvop0iySg5Cfi7yZDZ8brCgw@mail.gmail.com>

Hey,

I'm trying to serialise some data. I have a c++ class:
        class ostream
        {
            std::ostream& stream;
        public:
            ostream(std::ostream& s) : stream(s) {}

            void write(const char* bytes, Uint32 count)
            {
                stream.write(bytes, count);
            }
        };
As you can see, it only wraps std::ostream.

I call the python function like this:
ostream stream(file);
GameObject* obj = new GameObject(...);
                if(boost::python::override f = get_override("serialise"))
                    return f(boost::python::object(boost::python::ptr(obj)),
boost::python::object(boost::python::ptr(&stream)));

Then in the python script:
    def serialise(self, gameobj, file):
        if(gameobj in self.components):
            b = pickle.dumps(self.components[gameobj])
            # this is some testing code and dont really make sense because I
don't know what to do...
            file.write(str(len(str(b))), 10) # try to serialise an int that
tells the size of the byte-object
            print(len(str(b)))
            file.write(str(b), len(str(b))) # write the byte object
        return len(str(b)) + 10 # return amount of bytes we wrote

First i "pickle" a python object. I convert that byte-object to a string
since my write() function in c++ takes a const char*. But i'd like to pass
the byte-object directly since const char* is basically a byte, right?

Also, the other problem, when im about to read the serialised data I need to
know the size of the previously serialised data to unpickle it. I thoght
that I could to the c++ way and add a integer before the byte stream that
reveal how big it is. But How can i serialise an int from python that takes
up constant "byte-size" for an integer between lets say 0-10000!?

Any help on this one? Thanks!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100929/4846d673/attachment.html>

From simwarg at gmail.com  Wed Sep 29 14:11:20 2010
From: simwarg at gmail.com (Simon W)
Date: Wed, 29 Sep 2010 14:11:20 +0200
Subject: [C++-sig] TypeError: No to_python (by-value) converter found
 for C++ type: class std::basic_ostream<char, struct std::char_traits<char> >
In-Reply-To: <4CA30928.4010404@kormoran.net>
References: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>
	<4CA30928.4010404@kormoran.net>
Message-ID: <AANLkTi=YsyZkmRmovwEfUQAnBh76ZwH4bFygN=JcUzur@mail.gmail.com>

Thank you for the answer. Is there any way I can prevent it from copying the
stream object even though std::ostream::write() returns a std::ostream& ?

On Wed, Sep 29, 2010 at 11:38 AM, Jakub Zytka <kubol at kormoran.net> wrote:

>
> > I get this error:
> > *TypeError: No to_python (by-value) converter found for C++ type: class
> > std::basic_ostream<char,struct std::char_traits<char> >*
> >
> > This is the python function
> > *    def serialise(self, gameobj, file):
> >          if(gameobj in self.components):
> >              file.write("HEY", 3)*
> >
> > What's the problem here? Is is because the ostream argument is an
> reference?
> The problem is you are trying to copy a noncopyable object:
>
>
>              return class_<std::ostream, bases<std::ios>,
>> boost::noncopyable>("ostream", no_init)
>>
> ostream is noncopyable
>
>
>                  .def("write", write ,
>> return_value_policy<copy_non_const_reference>())
>>
> but you define write to return copy (of ostream)
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100929/9bb92fb3/attachment.html>

From kubol at kormoran.net  Wed Sep 29 16:02:09 2010
From: kubol at kormoran.net (Jakub Zytka)
Date: Wed, 29 Sep 2010 16:02:09 +0200
Subject: [C++-sig] TypeError: No to_python (by-value) converter found
 for C++ type: class std::basic_ostream<char, struct std::char_traits<char> >
In-Reply-To: <AANLkTi=YsyZkmRmovwEfUQAnBh76ZwH4bFygN=JcUzur@mail.gmail.com>
References: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>	<4CA30928.4010404@kormoran.net>
	<AANLkTi=YsyZkmRmovwEfUQAnBh76ZwH4bFygN=JcUzur@mail.gmail.com>
Message-ID: <4CA346E1.8070501@kormoran.net>

On 09/29/10 14:11, Simon W wrote:
> Thank you for the answer. Is there any way I can prevent it from copying the
> stream object even though std::ostream::write() returns a std::ostream& ?
You can use other return policy, or define one on your own.

http://www.boost.org/doc/libs/1_44_0/libs/python/doc/tutorial/doc/html/python/functions.html#python.call_policies

But honestly, I do not think it is the correct way to go. Read below.

 > I'm trying to serialise some data. I have a c++ class:
 >          class ostream
 >          {
 > ...
 > could to the c++ way and add a integer before the byte stream that reveal how
 > big it is. But How can i serialise an int from python that takes up constant
 > "byte-size" for an integer between lets say 0-10000!?
Everything can be done, but I have a different question:
I haven't followed all your previous emails, but do you *really* need to 
implement serialization code in python?
Somehow I feel it is very weird that on one hand you want to have serialization 
code in python, and yet use ostream.

What would be wrong with coding serialization purely in C++, and then providing 
appropriate pickle suite to python:
struct MyPickleSuite
{
// ...
boost::python::tuple MyPickleSuite::getstate(MyClass & object)
{
std::ostringstream oStream(ios_base::binary);
serialize(oStream, object);
return boost::python::make_tuple(oStream.str());
}

void MyPickleSuite::setstate(MyClass & object, boost::python::tuple state)
{
// verify tuple correctness etc.
std::string serializedData = extract<std::string>(state[0]);
std::istringstream iStream(serializedData, ios_base::binary);
serialize(iStream, object); // NOTE: you can use the same code for serialization 
and deserialization. stream type should determine actual behavior
}
}

That way you do not have to expose ostream to python at all. In fact if you 
wanted to switch to another type of stream (imagine that you have a custom 
stream implementation, which provides eg. type checking) it is cheap. You only 
change a few functions in your pickle suite, nothing more.

Even if you do really need to define serialization in python I see no reason to 
play with streams. I'd rather had a wrapper which hides underlying 
implementation (eg. std::streams) and operates on strings.

Please reconsider your design.

From wk at ipf.tuwien.ac.at  Wed Sep 29 17:02:57 2010
From: wk at ipf.tuwien.ac.at (Willi Karel)
Date: Wed, 29 Sep 2010 17:02:57 +0200
Subject: [C++-sig] Derivation from Python str
Message-ID: <4CA35521.3090403@ipf.tuwien.ac.at>

I'd like to derive from either Python's built-in 'str' or
boost::python::str, resulting in a class that provides additional
methods / member functions / attributes. The definitions of these
additional methods are implemented in C++. Is there any way to achieve
that? Please note that implicit conversions don't serve this goal, as
methods of 'str' should be part of the derived classes set of attributes
- e.g. the following Python code should be ok ('Derived' being the class
derived from Python's 'str'):

>>> Derived d
>>> d.startswith('prefix')

I see that Python's 'str' is immutable, which is acceptable for my purpose.

What I tried:

Including e.g. boost::python::str in the list of base classes yields a
run-time error in python:
boost::python::class_<Derived, boost::python::bases< boost::python::str
> >("Derived")
.def("foo",&Derived::foo)
;

Including std::string (which is built-in-converted to Python's 'str') in
the list of base classes yields a run-time assertion complaining about
the base class not having been wrapped. What I'd like to achieve anyway,
is derivation from Python's 'str', not std::string.

Thanks for any help!
Cheers, Willi.

From hans_meine at gmx.net  Wed Sep 29 18:01:43 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 29 Sep 2010 18:01:43 +0200
Subject: [C++-sig] Derivation from Python str
In-Reply-To: <4CA35521.3090403@ipf.tuwien.ac.at>
References: <4CA35521.3090403@ipf.tuwien.ac.at>
Message-ID: <201009291802.08576.hans_meine@gmx.net>

Am Mittwoch 29 September 2010, 17:02:57 schrieb Willi Karel:
> I'd like to derive from either Python's built-in 'str' or
> boost::python::str, resulting in a class that provides additional
> methods / member functions / attributes. [...]

AFAIK this is just an unsupported feature of the BPL.  I was (and would still 
be) interested in the same thing, only with list/dict being the base classes, 
but this does not seem to be possible.

From the outside, I would guess/hope that it's not too hard to fix, but having 
looked at various parts of boost::python in the past, I don't think that I 
will try it myself in the near future.

> Including e.g. boost::python::str in the list of base classes yields a
> run-time error in python: [...]

If it was supported, I would say that this would be the right approach/API.

Have a nice day,
  Hans

From talljimbo at gmail.com  Wed Sep 29 22:14:43 2010
From: talljimbo at gmail.com (Jim Bosch)
Date: Wed, 29 Sep 2010 13:14:43 -0700
Subject: [C++-sig] Derivation from Python str
In-Reply-To: <201009291802.08576.hans_meine@gmx.net>
References: <4CA35521.3090403@ipf.tuwien.ac.at>
	<201009291802.08576.hans_meine@gmx.net>
Message-ID: <4CA39E33.6080906@gmail.com>

On 09/29/2010 09:01 AM, Hans Meine wrote:
> Am Mittwoch 29 September 2010, 17:02:57 schrieb Willi Karel:
>> I'd like to derive from either Python's built-in 'str' or
>> boost::python::str, resulting in a class that provides additional
>> methods / member functions / attributes. [...]
>
> AFAIK this is just an unsupported feature of the BPL.  I was (and would still
> be) interested in the same thing, only with list/dict being the base classes,
> but this does not seem to be possible.
>
>> From the outside, I would guess/hope that it's not too hard to fix, but having
> looked at various parts of boost::python in the past, I don't think that I
> will try it myself in the near future.
>
>> Including e.g. boost::python::str in the list of base classes yields a
>> run-time error in python: [...]
>
> If it was supported, I would say that this would be the right approach/API.
>

If you *really* need to subclass Python's str type (so your objects pass 
an "isinstance(obj, str)" check, it is possible to do that in a way that 
is interoperable with Boost.Python (you'd be able to hold such an object 
in boost::python::str, for instance), but you'd have to use a lot of the 
raw Python C API to do it.  It would certainly be harder than just 
adding str-like methods to a regular Boost.Python class.

Jim Bosch

From wk at ipf.tuwien.ac.at  Thu Sep 30 03:21:32 2010
From: wk at ipf.tuwien.ac.at (Willi Karel)
Date: Thu, 30 Sep 2010 03:21:32 +0200
Subject: [C++-sig] Derivation from Python str
In-Reply-To: <4CA39E33.6080906@gmail.com>
References: <4CA35521.3090403@ipf.tuwien.ac.at>	<201009291802.08576.hans_meine@gmx.net>
	<4CA39E33.6080906@gmail.com>
Message-ID: <4CA3E61C.4010906@ipf.tuwien.ac.at>

Dear Hans and Jim,

thanks a lot for your replies!

I tried to wrap/def boost::python::str's methods for my derived class,
which is possible in a quite compact way using boost::function_types and
some templates. However, I quit my attempt when it came to operators -
too much work for now.

Cheers, Willi.

Jim Bosch wrote:
> On 09/29/2010 09:01 AM, Hans Meine wrote:
>> Am Mittwoch 29 September 2010, 17:02:57 schrieb Willi Karel:
>>> I'd like to derive from either Python's built-in 'str' or
>>> boost::python::str, resulting in a class that provides additional
>>> methods / member functions / attributes. [...]
>>
>> AFAIK this is just an unsupported feature of the BPL.  I was (and
>> would still
>> be) interested in the same thing, only with list/dict being the base
>> classes,
>> but this does not seem to be possible.
>>
>>> From the outside, I would guess/hope that it's not too hard to fix,
>>> but having
>> looked at various parts of boost::python in the past, I don't think
>> that I
>> will try it myself in the near future.
>>
>>> Including e.g. boost::python::str in the list of base classes yields a
>>> run-time error in python: [...]
>>
>> If it was supported, I would say that this would be the right
>> approach/API.
>>
> 
> If you *really* need to subclass Python's str type (so your objects pass
> an "isinstance(obj, str)" check, it is possible to do that in a way that
> is interoperable with Boost.Python (you'd be able to hold such an object
> in boost::python::str, for instance), but you'd have to use a lot of the
> raw Python C API to do it.  It would certainly be harder than just
> adding str-like methods to a regular Boost.Python class.
> 
> Jim Bosch
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig

-- 
                          __ ____ ____   Technische Universit?t Wien
   DI Wilfried Karel     /_//   // __/  Institut f?r Photogrammetrie
                        / // / // /_   und Fernerkundung (E122)
 wk at ipf.tuwien.ac.at   / // __// __/  Gu?hausstra?e 27-29
+43 1 58801 12244     /_//_/  /_/    A-1040 Wien

From simwarg at gmail.com  Thu Sep 30 16:38:50 2010
From: simwarg at gmail.com (Simon W)
Date: Thu, 30 Sep 2010 16:38:50 +0200
Subject: [C++-sig] TypeError: No to_python (by-value) converter found
 for C++ type: class std::basic_ostream<char, struct std::char_traits<char> >
In-Reply-To: <4CA346E1.8070501@kormoran.net>
References: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>
	<4CA30928.4010404@kormoran.net>
	<AANLkTi=YsyZkmRmovwEfUQAnBh76ZwH4bFygN=JcUzur@mail.gmail.com>
	<4CA346E1.8070501@kormoran.net>
Message-ID: <AANLkTik-cpn8fG5j3ipawtcbWAVktRQYonvq4Bah_nQf@mail.gmail.com>

I don't quite understand your approach to 100%. But you gave me another
idea. Here's how it looks when I call python function:

        Uint32 serialise(const GameObject* obj, std::ostream& file)
        {
            if(boost::python::override f = get_override("serialise"))
            {
                // call python function and ask to return a str() of the
pickled bytes
                boost::python::str pickled =
f(boost::python::object(boost::python::ptr(obj)));
                tuple tup = make_tuple(pickled);
                std::string s = extract<std::string>(tup[0]);
                Uint32 size = s.size();

                // serialise the size of pickled object
                file.write((char*)&size, sizeof(Uint32));

                 // serialise pickled bytes as string
                file.write(s.c_str(), size);
                return size + sizeof(Uint32);
            }
        }
        Uint32    Python::ComponentSystem::deserialise(std::istream& file,
const GameObject* obj)
        {
            if(boost::python::override f = get_override("deserialise"))
            {
                Uint32 size = 0;

                // we must know how many bytes to extract
                file.read((char*)&size, sizeof(Uint32));

                // OK, here starts the conversion, but Im not quite sure how
to do exactly ...
                char* sbuf = new char[size];
                file.read(sbuf, size);
                std::string sdata(sbuf);

                boost::python::object data(sdata); // ERROR:
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 80-85:
unsupported Unicode code range

                // send the chunk of bytes/string to the deserialise python
function and let it handle the restore of object
                f(boost::python::object(boost::python::ptr(obj)), data);
            }

            return  sizeof(Uint32) + size;
        }

And the python script:
    def serialise(self, gameobj):
        if(gameobj in self.components):
            return str(pickle.dumps(self.components[gameobj]))

        return 0

    def deserialise(self, gameobj, data):
        pass

On Wed, Sep 29, 2010 at 4:02 PM, Jakub Zytka <kubol at kormoran.net> wrote:

> On 09/29/10 14:11, Simon W wrote:
>
>> Thank you for the answer. Is there any way I can prevent it from copying
>> the
>> stream object even though std::ostream::write() returns a std::ostream& ?
>>
> You can use other return policy, or define one on your own.
>
>
> http://www.boost.org/doc/libs/1_44_0/libs/python/doc/tutorial/doc/html/python/functions.html#python.call_policies
>
> But honestly, I do not think it is the correct way to go. Read below.
>
> > I'm trying to serialise some data. I have a c++ class:
> >          class ostream
> >          {
> > ...
> > could to the c++ way and add a integer before the byte stream that reveal
> how
> > big it is. But How can i serialise an int from python that takes up
> constant
> > "byte-size" for an integer between lets say 0-10000!?
> Everything can be done, but I have a different question:
> I haven't followed all your previous emails, but do you *really* need to
> implement serialization code in python?
> Somehow I feel it is very weird that on one hand you want to have
> serialization code in python, and yet use ostream.
>
> What would be wrong with coding serialization purely in C++, and then
> providing appropriate pickle suite to python:
> struct MyPickleSuite
> {
> // ...
> boost::python::tuple MyPickleSuite::getstate(MyClass & object)
> {
> std::ostringstream oStream(ios_base::binary);
> serialize(oStream, object);
> return boost::python::make_tuple(oStream.str());
> }
>
> void MyPickleSuite::setstate(MyClass & object, boost::python::tuple state)
> {
> // verify tuple correctness etc.
> std::string serializedData = extract<std::string>(state[0]);
> std::istringstream iStream(serializedData, ios_base::binary);
> serialize(iStream, object); // NOTE: you can use the same code for
> serialization and deserialization. stream type should determine actual
> behavior
> }
> }
>
> That way you do not have to expose ostream to python at all. In fact if you
> wanted to switch to another type of stream (imagine that you have a custom
> stream implementation, which provides eg. type checking) it is cheap. You
> only change a few functions in your pickle suite, nothing more.
>
> Even if you do really need to define serialization in python I see no
> reason to play with streams. I'd rather had a wrapper which hides underlying
> implementation (eg. std::streams) and operates on strings.
>
> Please reconsider your design.
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20100930/76dc5347/attachment.html>

From simwarg at gmail.com  Thu Sep 30 17:41:57 2010
From: simwarg at gmail.com (Simon Warg)
Date: Thu, 30 Sep 2010 17:41:57 +0200
Subject: [C++-sig] TypeError: No to_python (by-value) converter found
	for C++ type: class std::basic_ostream<char,
	struct std::char_traits<char> >
In-Reply-To: <4CA346E1.8070501@kormoran.net>
References: <AANLkTimntNZ5Wp35bY0oESZ-tKd-J7jmqM4t_hOPMg2F@mail.gmail.com>
	<4CA30928.4010404@kormoran.net>
	<AANLkTi=YsyZkmRmovwEfUQAnBh76ZwH4bFygN=JcUzur@mail.gmail.com>
	<4CA346E1.8070501@kormoran.net>
Message-ID: <FD5FA514-9A10-4A68-B787-4D0CF04E220E@gmail.com>

And I forgot to tell you, the python class in an extension of a c++  
class. Hence I have the serialise and deserialise function defined in  
python.

// Simon

On 29 sep 2010, at 16.02, Jakub Zytka <kubol at kormoran.net> wrote:

> On 09/29/10 14:11, Simon W wrote:
>> Thank you for the answer. Is there any way I can prevent it from  
>> copying the
>> stream object even though std::ostream::write() returns a  
>> std::ostream& ?
> You can use other return policy, or define one on your own.
>
> http://www.boost.org/doc/libs/1_44_0/libs/python/doc/tutorial/doc/html/python/functions.html#python.call_policies
>
> But honestly, I do not think it is the correct way to go. Read below.
>
> > I'm trying to serialise some data. I have a c++ class:
> >          class ostream
> >          {
> > ...
> > could to the c++ way and add a integer before the byte stream that  
> reveal how
> > big it is. But How can i serialise an int from python that takes  
> up constant
> > "byte-size" for an integer between lets say 0-10000!?
> Everything can be done, but I have a different question:
> I haven't followed all your previous emails, but do you *really*  
> need to implement serialization code in python?
> Somehow I feel it is very weird that on one hand you want to have  
> serialization code in python, and yet use ostream.
>
> What would be wrong with coding serialization purely in C++, and  
> then providing appropriate pickle suite to python:
> struct MyPickleSuite
> {
> // ...
> boost::python::tuple MyPickleSuite::getstate(MyClass & object)
> {
> std::ostringstream oStream(ios_base::binary);
> serialize(oStream, object);
> return boost::python::make_tuple(oStream.str());
> }
>
> void MyPickleSuite::setstate(MyClass & object, boost::python::tuple  
> state)
> {
> // verify tuple correctness etc.
> std::string serializedData = extract<std::string>(state[0]);
> std::istringstream iStream(serializedData, ios_base::binary);
> serialize(iStream, object); // NOTE: you can use the same code for  
> serialization and deserialization. stream type should determine  
> actual behavior
> }
> }
>
> That way you do not have to expose ostream to python at all. In fact  
> if you wanted to switch to another type of stream (imagine that you  
> have a custom stream implementation, which provides eg. type  
> checking) it is cheap. You only change a few functions in your  
> pickle suite, nothing more.
>
> Even if you do really need to define serialization in python I see  
> no reason to play with streams. I'd rather had a wrapper which hides  
> underlying implementation (eg. std::streams) and operates on strings.
>
> Please reconsider your design.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig