From stefanrin at gmail.com  Sat May  2 14:43:17 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Sat, 2 May 2015 14:43:17 +0200
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
Message-ID: <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>

On Thu, Apr 2, 2015 at 9:12 PM, Peter Sch?ller <schueller.p at gmail.com> wrote:
> Dear all,
>
> In the hexhex project on github [1] we execute a python program using
> boost::python::exec_file, the respective code is in
> src/PythonPlugin.cpp in the function PythonPlugin::runPythonMain(...).
>
> In this python program I use subprocess.call() to run a shell
> application (actually graphviz).
>
> Sometimes it works, sometimes it hangs using 100% CPU. It seems to
> depend on how much the C++ program had to do before it executed the
> python part. In both cases the C++ program uses threads (is that
> relevant wrt. fork() in python? I found some hints online that it
> could be relevant).
>
> I traced it down to os.fork() in subprocess.py which returns in one
> case and does not return at all (neither parent nor child) in the
> other case.
>
> If I do strace -f on the process I get the following (the
> SIGPROF/rt_sigreturn/clone repeats until the disk is full):
>
> [pid 18646] clone(child_stack=0,
> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
> child_tidptr=0x7f96a5fa2a50) = ? ERESTARTNOINTR (To be restarted)
> [pid 18646] --- SIGPROF {si_signo=SIGPROF, si_code=SI_KERNEL} ---
> [pid 18646] rt_sigreturn()              = 56
> [pid 18646] clone(child_stack=0,
> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
> child_tidptr=0x7f96a5fa2a50) = ? ERESTARTNOINTR (To be restarted)
> [pid 18646] --- SIGPROF {si_signo=SIGPROF, si_code=SI_KERNEL} ---
> [pid 18646] rt_sigreturn()              = 56
> [pid 18646] clone(child_stack=0,
> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
> child_tidptr=0x7f96a5fa2a50) = ? ERESTARTNOINTR (To be restarted)
> [pid 18646] --- SIGPROF {si_signo=SIGPROF, si_code=SI_KERNEL} ---
> [pid 18646] rt_sigreturn()              = 56
> [pid 18646] clone(child_stack=0,
> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
> child_tidptr=0x7f96a5fa2a50) = ? ERESTARTNOINTR (To be restarted)
> [pid 18646] --- SIGPROF {si_signo=SIGPROF, si_code=SI_KERNEL} ---

There is no loop in Python's fork code, so it cannot really be blamed.
It sure looks like there is a pending signal which interferes for some
reason.

From schueller.p at gmail.com  Mon May  4 08:45:38 2015
From: schueller.p at gmail.com (=?UTF-8?Q?Peter_Sch=C3=BCller?=)
Date: Mon, 4 May 2015 09:45:38 +0300
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
 <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
Message-ID: <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>

On Sat, May 2, 2015 at 3:43 PM, Stefan Ring <stefanrin at gmail.com> wrote:
> On Thu, Apr 2, 2015 at 9:12 PM, Peter Sch?ller <schueller.p at gmail.com> wrote:
>> In the hexhex project on github [1] we execute a python program using
>> boost::python::exec_file, the respective code is in
>> src/PythonPlugin.cpp in the function PythonPlugin::runPythonMain(...).
>>
>> In this python program I use subprocess.call() to run a shell
>> application (actually graphviz).
>>
>> Sometimes it works, sometimes it hangs using 100% CPU. It seems to
>> depend on how much the C++ program had to do before it executed the
>> python part. In both cases the C++ program uses threads (is that
>> relevant wrt. fork() in python? I found some hints online that it
>> could be relevant).
>>
>> I traced it down to os.fork() in subprocess.py which returns in one
>> case and does not return at all (neither parent nor child) in the
>> other case.
>>
>> If I do strace -f on the process I get the following (the
>> SIGPROF/rt_sigreturn/clone repeats until the disk is full):
>>
>> [pid 18646] clone(child_stack=0,
>> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
>> child_tidptr=0x7f96a5fa2a50) = ? ERESTARTNOINTR (To be restarted)
>> [pid 18646] --- SIGPROF {si_signo=SIGPROF, si_code=SI_KERNEL} ---
>> [pid 18646] rt_sigreturn()              = 56
>> [pid 18646] clone(child_stack=0,
>> flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
>> child_tidptr=0x7f96a5fa2a50) = ? ERESTARTNOINTR (To be restarted)
>> [pid 18646] --- SIGPROF {si_signo=SIGPROF, si_code=SI_KERNEL} ---
>> [pid 18646] rt_sigreturn()              = 56
>> [pid 18646] clone(child_stack=0,
>
> There is no loop in Python's fork code, so it cannot really be blamed.
> It sure looks like there is a pending signal which interferes for some
> reason.

Thank you.

I have a signal handler for SIGINT in the program, could this interfere?
Or should I setup another signal handler?
Or is it necessary to setup signal handlers in a special way when
using boost::python?
Could you suggest a way to get more useful debugging information to
track down the problem?

Best,
Peter

-- 
http://www.peterschueller.com/

From stefanrin at gmail.com  Tue May  5 15:48:46 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Tue, 5 May 2015 15:48:46 +0200
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
 <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
 <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>
Message-ID: <CAAxjCEx7RJTFdc3c7ttr_P_oiEn3N1-hfAi0La2cgW-Kwdymvw@mail.gmail.com>

On Mon, May 4, 2015 at 8:45 AM, Peter Sch?ller <schueller.p at gmail.com> wrote:
> Could you suggest a way to get more useful debugging information to
> track down the problem?

I'd try to find out if there is a signal handler for signal 56 (I'm
guessing that's the signal number) and where it's coming from.

From stefanrin at gmail.com  Tue May  5 15:50:35 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Tue, 5 May 2015 15:50:35 +0200
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CAAxjCEx7RJTFdc3c7ttr_P_oiEn3N1-hfAi0La2cgW-Kwdymvw@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
 <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
 <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>
 <CAAxjCEx7RJTFdc3c7ttr_P_oiEn3N1-hfAi0La2cgW-Kwdymvw@mail.gmail.com>
Message-ID: <CAAxjCEzQJ6uEncbPMS+K7vZWXwK2psfKbSfYWZFUb-ceB+drpw@mail.gmail.com>

On Tue, May 5, 2015 at 3:48 PM, Stefan Ring <stefanrin at gmail.com> wrote:
> On Mon, May 4, 2015 at 8:45 AM, Peter Sch?ller <schueller.p at gmail.com> wrote:
>> Could you suggest a way to get more useful debugging information to
>> track down the problem?
>
> I'd try to find out if there is a signal handler for signal 56 (I'm
> guessing that's the signal number) and where it's coming from.

Actually, it's SIGPROF (27), as I just noticed.

From stefanrin at gmail.com  Tue May  5 16:00:40 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Tue, 5 May 2015 16:00:40 +0200
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CAAxjCEzQJ6uEncbPMS+K7vZWXwK2psfKbSfYWZFUb-ceB+drpw@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
 <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
 <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>
 <CAAxjCEx7RJTFdc3c7ttr_P_oiEn3N1-hfAi0La2cgW-Kwdymvw@mail.gmail.com>
 <CAAxjCEzQJ6uEncbPMS+K7vZWXwK2psfKbSfYWZFUb-ceB+drpw@mail.gmail.com>
Message-ID: <CAAxjCEz=mh_6=P7AmT5L7FMM_Pwb_B917zsm9O-0ndNykj+hVw@mail.gmail.com>

See also <http://caml.inria.fr/mantis/view.php?id=5893>. Maybe
RedHat's bugfix never made it into mainline, or the problem
reappeared.

From eu at doxos.eu  Wed May  6 07:27:01 2015
From: eu at doxos.eu (=?UTF-8?B?VsOhY2xhdiDFoG1pbGF1ZXI=?=)
Date: Wed, 06 May 2015 07:27:01 +0200
Subject: [C++-sig] shared_ptr and GIL crash in boost::python (patch)
Message-ID: <5549A625.2020007@doxos.eu>

Hi everybody,

I have been affected for years by the [1] bug: when shared_ptr for an 
object created in python is released in c++ in multi-threaded code, 
Python crashes -- because the shared_ptr_deleter does not lock the GIL 
before calling Python functions (as the object is destroyed).

The referenced bug also contains a 2-line patch; several people report 
that they have to hand-patch boost::python for their production code. In 
my case, I can work around the bug by locking the GIL around those few 
places where this scenario can occur (they are fortunately relatively 
few in my case), without recompiling boost.

I would like to ask someone to look at the patch if it is safe to be 
applied, and do so. It's been first identified 2 years ago. If I can do 
something on my part, or other reporters, I am willing to do whatever is 
necessary to expedite this.

Opinions?

Cheers, V?clav

[1] https://svn.boost.org/trac/boost/ticket/8290

From stefanrin at gmail.com  Thu May  7 17:27:31 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Thu, 7 May 2015 17:27:31 +0200
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CAAxjCEz=mh_6=P7AmT5L7FMM_Pwb_B917zsm9O-0ndNykj+hVw@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
 <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
 <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>
 <CAAxjCEx7RJTFdc3c7ttr_P_oiEn3N1-hfAi0La2cgW-Kwdymvw@mail.gmail.com>
 <CAAxjCEzQJ6uEncbPMS+K7vZWXwK2psfKbSfYWZFUb-ceB+drpw@mail.gmail.com>
 <CAAxjCEz=mh_6=P7AmT5L7FMM_Pwb_B917zsm9O-0ndNykj+hVw@mail.gmail.com>
Message-ID: <CAAxjCEzJQ_wFWdXTCN0K5FxMyKskx4Ba0uVwhDdq=hKANV5RyQ@mail.gmail.com>

On Tue, May 5, 2015 at 4:00 PM, Stefan Ring <stefanrin at gmail.com> wrote:
> See also <http://caml.inria.fr/mantis/view.php?id=5893>. Maybe
> RedHat's bugfix never made it into mainline, or the problem
> reappeared.

Completely unrelated to boost::python, but I guess you could just
block the SIGPROF signal before forking and unblock it afterwards. The
RHEL 5 kernel has apparently been patched with some sort of
workaround, but this has not made it into mainline or the later RHEL
branches.

From stefanrin at gmail.com  Sat May  9 13:31:25 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Sat, 9 May 2015 13:31:25 +0200
Subject: [C++-sig] shared_ptr and GIL crash in boost::python (patch)
In-Reply-To: <5549A625.2020007@doxos.eu>
References: <5549A625.2020007@doxos.eu>
Message-ID: <CAAxjCEwFFGXLw_6zkkio9JCahKpJZ-i9144zU3iesQy7E4YRow@mail.gmail.com>

On Wed, May 6, 2015 at 7:27 AM, V?clav ?milauer <eu at doxos.eu> wrote:
> I would like to ask someone to look at the patch if it is safe to be
> applied, and do so. It's been first identified 2 years ago. If I can do
> something on my part, or other reporters, I am willing to do whatever is
> necessary to expedite this.
>
> Opinions?

I guess it would be very prone to deadlocks in existing code. I, for
one, would not welcome such an unexpected change of behavior in a
minor version upgrade.

From schueller.p at gmail.com  Wed May 13 12:05:32 2015
From: schueller.p at gmail.com (=?UTF-8?Q?Peter_Sch=C3=BCller?=)
Date: Wed, 13 May 2015 13:05:32 +0300
Subject: [C++-sig] subprocess fork() sometimes hangs when called from
 within boost::python::exec_file
In-Reply-To: <CAAxjCEzJQ_wFWdXTCN0K5FxMyKskx4Ba0uVwhDdq=hKANV5RyQ@mail.gmail.com>
References: <CALomXVq72Byea8yG4to+LrjGF1qC3cRK9uiU7XJb33T=UZBo1g@mail.gmail.com>
 <CAAxjCEy_2wQy-wrfCFONmsnJOEd8+2h=aHq2-xZ2du_bwh4CSw@mail.gmail.com>
 <CALomXVqMzrRk6xWQPLw+asJ4f5gK+TFKDiForsR_KDyNGq=2tw@mail.gmail.com>
 <CAAxjCEx7RJTFdc3c7ttr_P_oiEn3N1-hfAi0La2cgW-Kwdymvw@mail.gmail.com>
 <CAAxjCEzQJ6uEncbPMS+K7vZWXwK2psfKbSfYWZFUb-ceB+drpw@mail.gmail.com>
 <CAAxjCEz=mh_6=P7AmT5L7FMM_Pwb_B917zsm9O-0ndNykj+hVw@mail.gmail.com>
 <CAAxjCEzJQ_wFWdXTCN0K5FxMyKskx4Ba0uVwhDdq=hKANV5RyQ@mail.gmail.com>
Message-ID: <CALomXVpBPF=3jBcxzGCfsvD4iGBsf1RDZc7CELg9Es4weg09_w@mail.gmail.com>

On Thu, May 7, 2015 at 6:27 PM, Stefan Ring <stefanrin at gmail.com> wrote:
> On Tue, May 5, 2015 at 4:00 PM, Stefan Ring <stefanrin at gmail.com> wrote:
>> See also <http://caml.inria.fr/mantis/view.php?id=5893>. Maybe
>> RedHat's bugfix never made it into mainline, or the problem
>> reappeared.
>
> Completely unrelated to boost::python, but I guess you could just
> block the SIGPROF signal before forking and unblock it afterwards. The
> RHEL 5 kernel has apparently been patched with some sort of
> workaround, but this has not made it into mainline or the later RHEL
> branches.
Unfortunately I cannot reproduce the problem anymore on the computer
it originally happened (on my other computer it never happened).

I noticed that I compiled with -pg which seems to be the reason for
the profiling signals that might prevent fork() to succeed.

So if someone else has the same problem, removing -pg from the
compilation might be sufficient to fix the problem.

Thank you for the support!
Peter

-- 
Peter SCH?LLER, Assistant Professor Dr
Marmara University, Computer Engineering Department
http://www.peterschueller.com/

From alain.miniussi at oca.eu  Tue May 12 16:44:27 2015
From: alain.miniussi at oca.eu (Alain Miniussi)
Date: Tue, 12 May 2015 16:44:27 +0200
Subject: [C++-sig] compilation problem with str
Message-ID: <555211CB.5020802@oca.eu>

Hi,

In:
http://www.boost.org/doc/libs/1_58_0/libs/python/doc/tutorial/doc/html/python/exposing.html#python.class_operators_special_functions

It is explained how to automagicaly get a str conversion from the 
operator<<(ostream&,blog const*) operator.

But, as explained in 
http://stackoverflow.com/questions/2828903/build-problems-when-adding-str-method-to-boost-python-c-class

It only works if the following
   using self_ns::str;
directive is present.

Now, I do not see any mention of that issue in the documentation. Did I 
overlooked it ? Or is it actually missing in the doc ? (or is it a but 
in the lib ?).

Thanks,

-- 
---
Alain

From David.Aldrich at EMEA.NEC.COM  Tue May 19 10:44:08 2015
From: David.Aldrich at EMEA.NEC.COM (David Aldrich)
Date: Tue, 19 May 2015 08:44:08 +0000
Subject: [C++-sig] undefined symbol: _ZTIN5boost6python15instance_holderE
Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B9753A4@EX10MBX02.EU.NEC.COM>

Hi

I have an application that uses Boost Python. It links and runs fine on Centos 5 systems but fails on an Ubuntu 14.04 LTS system.

The code that uses Boost Python is compiled using:

g++ -c -Wall -m64 -DPYTHON_ENABLE -I../Kernel -I/usr/include/python2.7 -I/usr/include/ -O3 main.cpp -o _gnuRelease/main.o

and linked into a static library using:


g++ -o _gnuRelease/mylib -Wl,-whole-archive,-export-dynamic ../Kernel/_gnuRelease/libKernel.a  -L/usr/lib/x86_64-linux-gnu  -lboost_python-py27 -lpython2.7 -Wl,--no-whole-archive -ldl

When the library is linked to my main program and run I see error:


undefined symbol: _ZTIN5boost6python15instance_holderE ( typeinfo for boost::python::instance_holder )

Please can someone suggest how to fix this?

Best regards

David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20150519/d7d43be2/attachment.html>

From mlang at delysid.org  Tue May 19 14:54:15 2015
From: mlang at delysid.org (Mario Lang)
Date: Tue, 19 May 2015 14:54:15 +0200
Subject: [C++-sig] undefined symbol: _ZTIN5boost6python15instance_holderE
In-Reply-To: <41302A7145AC054FA7A96CFD03835A0A0B9753A4@EX10MBX02.EU.NEC.COM>
 (David Aldrich's message of "Tue, 19 May 2015 08:44:08 +0000")
References: <41302A7145AC054FA7A96CFD03835A0A0B9753A4@EX10MBX02.EU.NEC.COM>
Message-ID: <87382swww8.fsf@fx.delysid.org>

David Aldrich <David.Aldrich at EMEA.NEC.COM> writes:

> I have an application that uses Boost Python. It links and runs fine on Centos 5 systems but fails on an Ubuntu 14.04 LTS system.
>
> The code that uses Boost Python is compiled using:
>
> g++ -c -Wall -m64 -DPYTHON_ENABLE -I../Kernel -I/usr/include/python2.7 -I/usr/include/ -O3 main.cpp -o _gnuRelease/main.o
>
> and linked into a static library using:
>
> g++ -o _gnuRelease/mylib -Wl,-whole-archive,-export-dynamic ../Kernel/_gnuRelease/libKernel.a  -L/usr/lib/x86_64-linux-gnu  -lboost_python-py27 -lpython2.7 -Wl,--no-whole-archive -ldl
>
> When the library is linked to my main program and run I see error:
>
> undefined symbol: _ZTIN5boost6python15instance_holderE ( typeinfo for boost::python::instance_holder )
>
> Please can someone suggest how to fix this?

Does your main program link with boost_python?
You didn't paste the link command of main, which would have been helpful
here.

Also, if you link with boost_python, does it match the Python version in
use (2 vs 3)?

-- 
CYa,
  ?????

From David.Aldrich at EMEA.NEC.COM  Wed May 20 10:09:03 2015
From: David.Aldrich at EMEA.NEC.COM (David Aldrich)
Date: Wed, 20 May 2015 08:09:03 +0000
Subject: [C++-sig] undefined symbol: _ZTIN5boost6python15instance_holderE
In-Reply-To: <87382swww8.fsf@fx.delysid.org>
References: <41302A7145AC054FA7A96CFD03835A0A0B9753A4@EX10MBX02.EU.NEC.COM>
 <87382swww8.fsf@fx.delysid.org>
Message-ID: <41302A7145AC054FA7A96CFD03835A0A0B975D67@EX10MBX02.EU.NEC.COM>

Hi

Thanks for your reply.

> Does your main program link with boost_python?
> You didn't paste the link command of main, which would have been helpful
> here.

I'm sorry, while sanitizing the command for posting here, I obscured some meaning.  The way the link process works is that the library includes a main(), we build the library with ar and then link it to form the executable:

ar: creating _gnuRelease/libKernel.a
g++ -o _gnuRelease/MyExecutable -Wl,-whole-archive,-export-dynamic ../Kernel/_gnuRelease/libKernel.a  -L/usr/lib/x86_64-linux-gnu  -lboost_python-py27 -lpython2.7 -Wl,--no-whole-archive -ldl

> Also, if you link with boost_python, does it match the Python version in use
> (2 vs 3)?

How would I check this please?

Best regards

David


From jraddison at gmail.com  Sat May 23 18:05:31 2015
From: jraddison at gmail.com (Jason Addison)
Date: Sat, 23 May 2015 10:05:31 -0600
Subject: [C++-sig] Returning values to Python in C++ reference arguments
Message-ID: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>

How can results be returned in function arguments?

I've include my example C++ extension and Python code below.

I've tried, what I think, are some obvious approaches (Python objects,
ctypes), though none have worked.

It seems like this should be doable.

If the python code owns, say, a c_int, I would thing it is mostly safe
to pass it to c++ to be modified.

This seems like it should be a common question, though I can't find it
being explicitly addressed in the documentation

All of the call policy stuff seems to only apply to the function result.

Can the policy stuff be applied to the function arguments?

If so, how would that work with my example?

Thanks,
Jason


my_module.cpp
>>>>
/*
clang++ -c -I/usr/include/python2.7 my_module.cpp
clang++ -dynamiclib -o my_module.so -lboost_python -lpython2.7
my_module.o
 */

#include <boost/python.hpp>

struct Foo {
    void bar(int &a, double &b) {
        a = 12;
        b = 6.2832;
    }
};

using namespace boost::python;

BOOST_PYTHON_MODULE(my_module)
{
    class_<Foo>("Foo", init<>())
        .def("bar", &Foo::bar)
    ;
}
<<<<


byref.py
>>>>
from ctypes import *
import my_module

f = my_module.Foo()

a = 0
b = 0.

# py objects
'''
f.bar(a, b)

Traceback (most recent call last):
  File "byref.py", line 8, in <module>
    f.bar(a, b)
Boost.Python.ArgumentError: Python argument types in
    Foo.bar(Foo, int, float)
did not match C++ signature:
    bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''

x = c_int(0)
y = c_double(0)

# as is
'''
f.bar(x, y)

Traceback (most recent call last):
  File "byref.py", line 25, in <module>
    f.bar(x, y)
Boost.Python.ArgumentError: Python argument types in
    Foo.bar(Foo, c_int, c_double)
did not match C++ signature:
    bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''

# ctype.byref
'''
f.bar(byref(x), byref(y))

Traceback (most recent call last):
  File "byref.py", line 36, in <module>
    f.bar(byref(x), byref(y))
Boost.Python.ArgumentError: Python argument types in
    Foo.bar(Foo, CArgObject, CArgObject)
did not match C++ signature:
    bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''

# ctype.pointer
'''
f.bar(pointer(x), pointer(y))

Traceback (most recent call last):
  File "byref.py", line 48, in <module>
    f.bar(pointer(x), pointer(y))
Boost.Python.ArgumentError: Python argument types in
    Foo.bar(Foo, LP_c_int, LP_c_double)
did not match C++ signature:
    bar(Foo {lvalue}, int {lvalue}, double {lvalue})
'''
<<<<

From trigves at yahoo.com  Mon May 25 11:39:30 2015
From: trigves at yahoo.com (Trigve Siver)
Date: Mon, 25 May 2015 09:39:30 +0000 (UTC)
Subject: [C++-sig] Returning values to Python in C++ reference arguments
In-Reply-To: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
References: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
Message-ID: <691722437.1536785.1432546770301.JavaMail.yahoo@mail.yahoo.com>



----- Original Message -----
> From: Jason Addison <jraddison at gmail.com>
> To: cplusplus-sig at python.org
> Cc: 
> Sent: Saturday, May 23, 2015 6:05 PM
> Subject: [C++-sig] Returning values to Python in C++ reference arguments
> 
> How can results be returned in function arguments?
> 


I don't think that you can return arguments by reference/pointers in python. You could theoretically pass a mutable sequence (list) and push the objects there.

But the common approach is to return tuple.

From nikolay.mladenov at gmail.com  Mon May 25 20:16:49 2015
From: nikolay.mladenov at gmail.com (Nikolay Mladenov)
Date: Mon, 25 May 2015 14:16:49 -0400
Subject: [C++-sig] Returning values to Python in C++ reference arguments
In-Reply-To: <691722437.1536785.1432546770301.JavaMail.yahoo@mail.yahoo.com>
References: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
 <691722437.1536785.1432546770301.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CACbdn-+PDOZYu4UX2RiVYGkcwVj0Y3VRT8yWj9KgBR4e3LUpsQ@mail.gmail.com>

You have to do something like that

using namespace boost::python;

BOOST_PYTHON_MODULE(my_module)
{
    struct wrapper{
       static tuple bar(Foo & f){
           double a, b;
           f.bar(a,b);
           return make_tuple(a,b);
       }
    };
    class_<Foo>("Foo", init<>())
        .def("bar", &wrapper::bar)
    ;
}

and in python it will be:

>>> a,b = Foo().bar()

On Mon, May 25, 2015 at 5:39 AM, Trigve Siver via Cplusplus-sig <
cplusplus-sig at python.org> wrote:

>
>
> ----- Original Message -----
> > From: Jason Addison <jraddison at gmail.com>
> > To: cplusplus-sig at python.org
> > Cc:
> > Sent: Saturday, May 23, 2015 6:05 PM
> > Subject: [C++-sig] Returning values to Python in C++ reference arguments
> >
> > How can results be returned in function arguments?
> >
>
>
> I don't think that you can return arguments by reference/pointers in
> python. You could theoretically pass a mutable sequence (list) and push the
> objects there.
>
> But the common approach is to return tuple.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> https://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20150525/bcf9cbb9/attachment.html>

From Holger.Joukl at LBBW.de  Tue May 26 09:41:28 2015
From: Holger.Joukl at LBBW.de (Holger Joukl)
Date: Tue, 26 May 2015 09:41:28 +0200
Subject: [C++-sig] Returning values to Python in C++ reference arguments
In-Reply-To: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
References: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
Message-ID: <OF293F286D.3EA223F1-ONC1257E51.0025EBA5-C1257E51.002A400E@LBBW.de>


> Von: Jason Addison
>
> How can results be returned in function arguments?
>
> I've include my example C++ extension and Python code below.
>
> I've tried, what I think, are some obvious approaches (Python objects,
> ctypes), though none have worked.
>
> It seems like this should be doable.

As already mentioned, you can use helper functions and return tuples.

Or you could expose custom "ref-object" classes to Python, s.th. along the
lines of:

// ref_object.hpp
#if !defined REF_OBJECT
#define REF_OBJECT


#include <boost/python.hpp>
#include <algorithm>


namespace bp = boost::python;

namespace refob {


// Basic RefObject
template<typename T>
struct RefObject {
public:
    T t;

    RefObject();
    // callable operator
    T operator()();

    // conversion functions: implicit castability
    operator T&() { return t; }
};


// RefObject for handling C-style arrays
// This provides an operator() that allows access to
// a list of values.
// NOTE: If numElements > the actual stored number of elements then
// THIS WILL CRASH! It is essential to correctly apply set_num() after
// this has been used as a function argument, in some thin wrappers
// There is only some very rudimentary safety in that _numElements is
// initialized to 0
// I really don't see a better way to make this safer
template<typename T>
struct RefObject<T const *> {
    T const * t;

    RefObject();
    // callable operator
    bp::list operator()(unsigned int numElements=0);

    // conversion functions: implicit castability
    operator T const *&() { return t; }

    void set_num(unsigned int const & numElements);

private:
    unsigned int _numElements;
};


// RefObject specialization for void*
template<>
struct RefObject<void *> {
    void* t;

    RefObject();
    // callable operator
    void* operator()();
};


// RefObject specialization for void const* (where we cast away const for
the
// operator() call result to make boost work automagically
template<>
struct RefObject<void const *> {
    void const * t;

    RefObject();
    // callable operator
    void* operator()();
};


// RefObject specialization for char const* which boost.python will
automatically
// handle as a python string (necessary to distingish from the C-style
array
// handling RefObject, see above
template<>
struct RefObject<char const *> {
    char const * t;

    RefObject();
    // callable operator
    char const* operator()();
};


// Is inlining the way to go to avoid multiply defined symbols here?
// Note: Remember that
//  * a specialization is not a template but a concrete (member function in
//     this case)
//  * as ref_object.hpp are
//    #included from every module there will be multiple definitions (one
//    in each compilation unit)
//  ==> linker chokes
// Using inline this can be avoided
// See
// http://msdn.microsoft.com/en-us/magazine/cc163769.aspx
// http://www.parashift.com/c++-faq-lite/inline-functions.html
// for background

template <typename T>
RefObject<T>::RefObject(): t() {
    //std::cout << "RefObject<" << typeid(T).name() << ">::RefObject()" <<
std::endl;
}


template <typename T>
RefObject<T const *>::RefObject(): t(), _numElements(0) {
    //std::cout << "RefObject<" << typeid(T).name() << " const
*>::RefObject()" << std::endl;
}


inline RefObject<void *>::RefObject(): t() {
    //std::cout << "RefObject<void *>::RefObject()" << std::endl;
}


inline RefObject<void const *>::RefObject(): t() {
    //std::cout << "RefObject<void const *>::RefObject()" << std::endl;
}


inline RefObject<char const *>::RefObject(): t() {
    //std::cout << "RefObject<char const *>::RefObject()" << std::endl;
}


template <typename T>
T RefObject<T>::operator()()
{
    // Do we need to care about pointer data issues here?
    return t;
}


template<typename T>
bp::list RefObject<T const *>::operator()(unsigned int numElements)
{
    bp::list list_object;

    unsigned int maxElements = std::min(numElements, _numElements);
    for (unsigned int i=0; i < maxElements; i++) {
        bp::object item(*t);
        list_object.append(item);
    }
    return list_object;
}


inline void* RefObject<void const *>::operator()()
{
    void* non_const_ptr = const_cast<void*>(t);
    //std::cout << "returning non-const void* t" << std::endl;
    return non_const_ptr;
}


inline void* RefObject<void *>::operator()()
{
    return t;
}


inline char const* RefObject<char const *>::operator()()
{
    return t;
}


template<typename T>
void RefObject<T const *>::set_num(unsigned int const & numElements)
{
    _numElements = numElements;
}


} // namespace refob

#endif // #if !defined REF_OBJECT


And you might expose those to Python:

// ref_object.cpp
#include <boost/python.hpp>
#include "ref_object.hpp"


namespace bp = boost::python;


namespace refob
{


// dummy function, expose this to force creation of void* converters in
boost registry
void* void_ptr_from_void_ptr(void* void_ptr) {
    return void_ptr;
}


void export_reference_type_classes()
{

//     // dummy function registration to force creation of void* converters
//     // (not necessary if there is an exposed function returning void*)
//     bp::def("void_ptr_from_void_ptr", &void_ptr_from_void_ptr,
//             bp::return_value_policy<bp::return_opaque_pointer>());

    // const-pointer types

    // 1. char

    // 1.1: const char*
    bp::class_<RefObject<const char*>, boost::noncopyable>
("const_char_ptr_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<const char*>::operator())
        ;


    // 2. numeric types (float/double/int)

    // Note: signed and unsigned char are *numeric* types usage-wise
    // const signed char*
//     bp::class_<RefObject<const signed char*>, boost::noncopyable>
("const_signed_char_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const signed char*>::operator())
//         ;


//     // const unsigned char*
//     bp::class_<RefObject<const unsigned char*>, boost::noncopyable>
("const_unsigned_char_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const unsigned char*>::operator())
//         ;

    // const double*
    bp::class_<RefObject<const double*>, boost::noncopyable>
("const_double_ptr_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<const double*>::operator())
        ;
    bp::implicitly_convertible< RefObject<const double*>, double const * >
();

//     // const float*
//     bp::class_<RefObject<const float*>, boost::noncopyable>
("const_float_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const float*>::operator())
//         ;

//     // const short*
//     bp::class_<RefObject<const short*>, boost::noncopyable>
("const_short_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const short*>::operator())
//         ;

//     // const unsigned short*
//     bp::class_<RefObject<const unsigned short*>, boost::noncopyable>
("const_unsigned_short_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const unsigned short*>::operator())
//         ;

//     // const int*
//     bp::class_<RefObject<const int*>, boost::noncopyable>
("const_int_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const int*>::operator())
//         ;

//     // const unsigned int*
//     bp::class_<RefObject<const unsigned int*>, boost::noncopyable>
("const_unsigned_int_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const unsigned int*>::operator())
//         ;


//     // const long long*
//     bp::class_<RefObject<const long long*>, boost::noncopyable>
("const_long_long_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const long long*>::operator())
//         ;

//     // const unsigned long long*
//     bp::class_<RefObject<const unsigned long long*>, boost::noncopyable>
("const_unsigned_long_long_ptr_ref")
//         .def(bp::init<>())
//         .def("__call__", &RefObject<const unsigned long long*>::operator
())
//         ;


    // 3. void

    // const void*
    bp::class_<RefObject<const void*>, boost::noncopyable>
("const_void_ptr_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<const void*>::operator(),
             bp::return_value_policy<bp::return_opaque_pointer>())
        ;



    // NON-const-pointer types:

    // void*
    bp::class_<RefObject<void*>, boost::noncopyable>("void_ptr_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<void*>::operator(),
             bp::return_value_policy<bp::return_opaque_pointer>())
        ;


    // Basic types

    // 1. char types

    // char
    bp::class_<RefObject<char>, boost::noncopyable>("char_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<char>::operator())
        ;


    // 2. numeric types

    // Note: signed and unsigned char are *numeric* types usage-wise
    // const signed char*
    // signed char
    bp::class_<RefObject<signed char>, boost::noncopyable>
("signed_char_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<signed char>::operator())
        ;

    // unsigned char
    bp::class_<RefObject<unsigned char>, boost::noncopyable>
("unsigned_char_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<unsigned char>::operator())
        ;

    // double
    bp::class_<RefObject<double>, boost::noncopyable>("double_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<double>::operator())
        ;

    // float
    bp::class_<RefObject<float>, boost::noncopyable>("float_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<float>::operator())
        ;

    // short
    bp::class_<RefObject<short>, boost::noncopyable>("short_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<short>::operator())
        ;

    // unsigned short
    bp::class_<RefObject<unsigned short>, boost::noncopyable>
("unsigned_short_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<unsigned short>::operator())
        ;

    // int
    bp::class_<RefObject<int>, boost::noncopyable>("int_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<int>::operator())
        ;

    // unsigned int
    bp::class_<RefObject<unsigned int>, boost::noncopyable>
("unsigned_int_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<unsigned int>::operator())
        ;
    bp::implicitly_convertible< RefObject<unsigned int>, unsigned int >();

    // long long
    bp::class_<RefObject<long long>, boost::noncopyable>("long_long_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<long long>::operator())
        ;

    // unsigned long long
    bp::class_<RefObject<unsigned long long>, boost::noncopyable>
("unsigned_long_long_ref")
        .def(bp::init<>())
        .def("__call__", &RefObject<unsigned long long>::operator())
        ;

};

} // namespace refob


Add to your module (or consider putting into a separate extension module):


namespace refob {
    //forward declaration must be in appropriate namespace
    void export_reference_type_classes(); // helpers/ref_object.cpp
}
BOOST_PYTHON_MODULE(my_module) {

        refob::export_reference_type_classes();
...

You'd then have to use the specialized ref objects as function arguments,
much
like what you tried with the ctypes stuff:

import my_module
f = my_module.Foo()
a = my_module.int_ref(0)
b = my_module.double_ref(0)
f.bar(a, b)

print "a result:", a()

If such effort makes sense depends on how "reference-call-littered"
the C++ API you wrap is.

Note: I hand-edited the original code a little to remove some irrelevant
stuff
and didn't try to compile and run it afterwards. But you get the idea.

Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart


From jraddison at gmail.com  Mon May 25 21:42:43 2015
From: jraddison at gmail.com (Jason Addison)
Date: Mon, 25 May 2015 13:42:43 -0600
Subject: [C++-sig] Returning values to Python in C++ reference arguments
In-Reply-To: <691722437.1536785.1432546770301.JavaMail.yahoo@mail.yahoo.com>
References: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
 <691722437.1536785.1432546770301.JavaMail.yahoo@mail.yahoo.com>
Message-ID: <CAA0wQb1F+gqyD_VuvFn3Y7MXnUO6kQo6QJNMmzpxA_OLAzvkDw@mail.gmail.com>

On Mon, May 25, 2015 at 3:39 AM, Trigve Siver via Cplusplus-sig
<cplusplus-sig at python.org> wrote:
>
>
>>
>> How can results be returned in function arguments?
>>
>
>
> I don't think that you can return arguments by reference/pointers in python. You could theoretically pass a mutable sequence (list) and push the objects there.
>

You can return results via function arguments from C to Python (see code below).

Why is something similar impossible with Boost Python?

my_module2.c
>>>>
/*
clang -c -I/usr/include/python2.7 my_module2.c
clang -dynamiclib -o my_module2.so -lpython2.7 my_module2.o
 */

void foo_bar(int *a, double *b) {
    *a = 12;
    *b = 6.2832;
}
<<<<

byref2.py
>>>>
from ctypes import *
my_module2 = CDLL('my_module2.so')

x = c_int(0)
y = c_double(0)

my_module2.foo_bar(byref(x), byref(y))

assert(x.value == 12)
assert(y.value == 6.2832)
<<<<

> But the common approach is to return tuple.

From Holger.Joukl at LBBW.de  Tue May 26 14:14:24 2015
From: Holger.Joukl at LBBW.de (Holger Joukl)
Date: Tue, 26 May 2015 14:14:24 +0200
Subject: [C++-sig] Returning values to Python in C++ reference arguments
In-Reply-To: <CAA0wQb1F+gqyD_VuvFn3Y7MXnUO6kQo6QJNMmzpxA_OLAzvkDw@mail.gmail.com>
References: <CAA0wQb0kYeVNFua-8LHd_OQYwsV6huQ42b48O+_45VMtUps30Q@mail.gmail.com>
 <691722437.1536785.1432546770301.JavaMail.yahoo@mail.yahoo.com>
 <CAA0wQb1F+gqyD_VuvFn3Y7MXnUO6kQo6QJNMmzpxA_OLAzvkDw@mail.gmail.com>
Message-ID: <OF152F22E7.F38A36FD-ONC1257E51.0042E39B-C1257E51.00433CB9@LBBW.de>

Hi,

> Von: Jason Addison
> You can return results via function arguments from C to Python (see
> code below).
>
> Why is something similar impossible with Boost Python?
>
> my_module2.c
> >>>>
> /*
> clang -c -I/usr/include/python2.7 my_module2.c
> clang -dynamiclib -o my_module2.so -lpython2.7 my_module2.o
>  */
>
> void foo_bar(int *a, double *b) {
>     *a = 12;
>     *b = 6.2832;
> }
> <<<<
>
> byref2.py
> >>>>
> from ctypes import *
> my_module2 = CDLL('my_module2.so')
>

Because you don't create Python built-in int or float objects here:

> x = c_int(0)
> y = c_double(0)

What does type(x) and type(y) give you?

> my_module2.foo_bar(byref(x), byref(y))
>
> assert(x.value == 12)
> assert(y.value == 6.2832)
> <<<<

(unrelated sidenote: it's probably not the best idea to test float/double
equality)

Best regards
Holger

Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart


From christoff.kok at ex-mente.co.za  Thu May 28 09:29:39 2015
From: christoff.kok at ex-mente.co.za (Christoff Kok)
Date: Thu, 28 May 2015 09:29:39 +0200
Subject: [C++-sig] Boost.Python C++ object reference in Python: unexpected
	behaviour.
Message-ID: <CAJ8CMJ_U3EQn5VAYbkaTXDBuqZajAOsqEroBM-AXByX_Y-kfUQ@mail.gmail.com>

Hi,

I am having an issue with Boost.Python with a very simple use case.

I am returning a reference to an object, and it seems that my python object
looses its C++ object's reference at a stage for some reason.

Please see my *example* below reproducing this issue.

*C++ Code:*

#include <iostream>
#include <vector>
#include <string>
#include <cmath>
#include <boost/python.hpp>
#include <boost/python/suite/indexing/vector_indexing_suite.hpp>

class Car {
public:
    Car(std::string name) : m_name(name) {}

    bool operator==(const Car &other) const {
        return m_name == other.m_name;
    }

    std::string GetName() { return m_name; }
private:
    std::string m_name;
};

class Factory {
public:
    Factory(std::string name) : m_name(name) {}

    bool operator==(const Factory &other) const {
        return m_name == other.m_name
            && m_car_list == other.m_car_list;
    }

    Car& create_car(std::string name)
    {
        m_car_list.emplace_back(Car(name));
        return m_car_list.back();
    }

    std::string GetName() { return m_name; }
    std::vector<Car>& GetCarList() { return m_car_list;}
private:
    std::string m_name;
    std::vector<Car> m_car_list;
};

class Manufacturer {
public:
    Manufacturer(std::string name) : m_name(name) {}

    bool operator==(const Manufacturer &other) const {
        return m_name == other.m_name
            && m_factory_list == other.m_factory_list;
    }

    Factory& create_factory(std::string name)
    {
        m_factory_list.emplace_back(Factory(name));
        return m_factory_list.back();
    }

    std::string GetName() { return m_name; }
    std::vector<Factory>& GetFactoryList() { return m_factory_list;}
private:
    std::string m_name;
    std::vector<Factory> m_factory_list;
};

BOOST_PYTHON_MODULE(carManufacturer)
{
    using namespace boost::python;
    class_<Manufacturer>("Manufacturer", init<std::string>())
        .add_property("factory_list",
make_function(&Manufacturer::GetFactoryList,
return_internal_reference<1>()))
        .add_property("name", &Manufacturer::GetName)
        .def("create_factory", &Manufacturer::create_factory,
return_internal_reference<>());
    class_<Factory>("Factory", init<std::string>())
        .add_property("car_list", make_function(&Factory::GetCarList,
return_internal_reference<1>()))
        .add_property("name", &Factory::GetName)
        .def("create_car", &Factory::create_car,
return_internal_reference<>());
    class_<Car>("Car", init<std::string>())
        .add_property("name", &Car::GetName);

    class_<std::vector<Factory> >("FactoryList")
        .def(vector_indexing_suite<std::vector<Factory> >());
    class_<std::vector<Car> >("Car")
        .def(vector_indexing_suite<std::vector<Car> >());
}


*Python Code:*

import sys
sys.path[:0] = [r"bin\Release"]

from carManufacturer import *

vw = Manufacturer("VW")
vw_bra_factory = vw.create_factory("Brazil Factory")
beetle = vw_bra_factory.create_car("Beetle69")

if vw_bra_factory is vw.factory_list[0]:
    print("equal.")
else:
    print("NOT EQUAL")
print("## I expected them to be the same reference..?")


print("vw_bra_factory Car List size : " + str(len(vw_bra_factory.car_list)))
print("Actual Car List size         : " +
str(len(vw.factory_list[0].car_list)))
print("## This still works. Maybe the python objects differ, but refer to
the same C++ object. I can live with that.")

vw_sa_factory = vw.create_factory("South Africa Factory")
print("vw_bra_factory Car List size : " + str(len(vw_bra_factory.car_list)))
print("Actual Car List size         : " +
str(len(vw.factory_list[0].car_list)))
print("## .. what? why? brazil py object has no cars now? I don't get it. I
can't have any of that.")

print("## What will happen if I create another car in the brazil factory?")
combi = vw_bra_factory.create_car("Hippie van")
print("vw_bra_factory Car List size : " + str(len(vw_bra_factory.car_list)))
print("Actual Car List size         : " +
str(len(vw.factory_list[0].car_list)))

print("## And another.")
citi_golf = vw_bra_factory.create_car("Citi golf")
print("vw_bra_factory Car List size : " + str(len(vw_bra_factory.car_list)))
print("Actual Car List size         : " +
str(len(vw.factory_list[0].car_list)))
print("## 'vw_bra_factory' must have lost its C++ reference it had to
'vw.factory_list[0]' when I created a new factory. Why?")


*Python Output:*

NOT EQUAL
*## I expected them to be the same reference..?*
vw_bra_factory Car List size : 1
Actual Car List size         : 1
*## This still works. Maybe the python objects differ, but refer to the
same C++ object. I can live with that.*
vw_bra_factory Car List size : 0
Actual Car List size         : 1
*## .. what? why? brazil py object has no cars now? I don't get it. I can't
have any of that.*
*## What will happen if I create another car in the brazil factory?*
vw_bra_factory Car List size : 1
Actual Car List size         : 1
*## And another.*
vw_bra_factory Car List size : 2
Actual Car List size         : 1
*## 'vw_bra_factory' must have lost its C++ reference it had to
'vw.factory_list[0]' when I created a new factory. Why?*

This is just an example made to reproduce my real work's problem in a
presentable way. In my real work, python crashes
after I create a second "factory" and try to add a "car" to the first
"factory".
The crash occurs in C++ "create_car" method whan trying to access the
"factory"'s "car" list.

Does anyone have insight as to what the problem is? Any useful input will
be greatly appreciated.

Greetings,
Christoff
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20150528/0df75403/attachment.html>

From stefan at seefeld.name  Fri May 29 16:28:46 2015
From: stefan at seefeld.name (Stefan Seefeld)
Date: Fri, 29 May 2015 10:28:46 -0400
Subject: [C++-sig] C++ copy construction and Python object copies
Message-ID: <5568779E.9030209@seefeld.name>

Hello,

Python's copy module allows for objects to be copied. The protocol for
this will look up special method __copy__. It seems to me that this
would trivially work for C++ objects providing a copy-constructor.
However, the copy-constructor isn't automatically bound to __copy__.
While I can certainly add that in user-code, I wonder why this isn't
done by Boost.Python itself.
Does anyone know the reasons for this ? Would it seem useful to add that
feature ?

		Stefan


-- 

      ...ich hab' noch einen Koffer in Berlin...


From amohr at pixar.com  Sat May 30 00:48:00 2015
From: amohr at pixar.com (Alex Mohr)
Date: Fri, 29 May 2015 15:48:00 -0700
Subject: [C++-sig] C++ copy construction and Python object copies
In-Reply-To: <5568779E.9030209@seefeld.name>
References: <5568779E.9030209@seefeld.name>
Message-ID: <5568ECA0.70703@pixar.com>

On 5/29/2015 7:28 AM, Stefan Seefeld wrote:
> Python's copy module allows for objects to be copied. The protocol for
> this will look up special method __copy__. It seems to me that this
> would trivially work for C++ objects providing a copy-constructor.
> However, the copy-constructor isn't automatically bound to __copy__.
> While I can certainly add that in user-code, I wonder why this isn't
> done by Boost.Python itself.
> Does anyone know the reasons for this ? Would it seem useful to add that
> feature ?

The __copy__ method's intended semantics are to produce a "shallow" copy 
while the __deepcopy__ method is meant to produce a "deep" copy.  Given 
a C++ class with a copy ctor, it's hard to know which is more 
appropriate.  For instance, copying a shared_ptr<T> is like a "shallow" 
copy but copying a vector<pair<int, float>> is like a deep copy.  On the 
other hand copying a vector<shared_ptr<T>> is more like a shallow copy.

I wouldn't mind an opt-in convenience utility that adds a __copy__ or 
__deepcopy__ method, but given the semantic subtlety, I think trying to 
do it automatically is dicey.

Also, adding a feature to do it automatically could potentially 
interfere with existing binding code that manually provides 
__copy__/__deepcopy__ methods.

Alex


From stefan at seefeld.name  Sat May 30 00:57:29 2015
From: stefan at seefeld.name (Stefan Seefeld)
Date: Fri, 29 May 2015 18:57:29 -0400
Subject: [C++-sig] C++ copy construction and Python object copies
In-Reply-To: <5568ECA0.70703@pixar.com>
References: <5568779E.9030209@seefeld.name> <5568ECA0.70703@pixar.com>
Message-ID: <5568EED9.4010206@seefeld.name>

On 29/05/15 06:48 PM, Alex Mohr wrote:
> On 5/29/2015 7:28 AM, Stefan Seefeld wrote:
>> Python's copy module allows for objects to be copied. The protocol for
>> this will look up special method __copy__. It seems to me that this
>> would trivially work for C++ objects providing a copy-constructor.
>> However, the copy-constructor isn't automatically bound to __copy__.
>> While I can certainly add that in user-code, I wonder why this isn't
>> done by Boost.Python itself.
>> Does anyone know the reasons for this ? Would it seem useful to add that
>> feature ?
>
> The __copy__ method's intended semantics are to produce a "shallow"
> copy while the __deepcopy__ method is meant to produce a "deep" copy. 
> Given a C++ class with a copy ctor, it's hard to know which is more
> appropriate.  For instance, copying a shared_ptr<T> is like a
> "shallow" copy but copying a vector<pair<int, float>> is like a deep
> copy.  On the other hand copying a vector<shared_ptr<T>> is more like
> a shallow copy.
>
> I wouldn't mind an opt-in convenience utility that adds a __copy__ or
> __deepcopy__ method, but given the semantic subtlety, I think trying
> to do it automatically is dicey.

That's a fair point.

> Also, adding a feature to do it automatically could potentially
> interfere with existing binding code that manually provides
> __copy__/__deepcopy__ methods.

True, but that could be mitigated by always letting an explicit addition
override the default.

    Stefan

-- 

      ...ich hab' noch einen Koffer in Berlin...


From trigves at yahoo.com  Sun May 31 09:16:57 2015
From: trigves at yahoo.com (Trigve Siver)
Date: Sun, 31 May 2015 07:16:57 +0000 (UTC)
Subject: [C++-sig] C++ copy construction and Python object copies
In-Reply-To: <5568EED9.4010206@seefeld.name>
References: <5568EED9.4010206@seefeld.name>
Message-ID: <1231436015.1934490.1433056617992.JavaMail.yahoo@mail.yahoo.com>



> From: Stefan Seefeld <stefan at seefeld.name>

> To: cplusplus-sig at python.org
> Cc: 
> Sent: Saturday, May 30, 2015 12:57 AM
> Subject: Re: [C++-sig] C++ copy construction and Python object copies
> 
> On 29/05/15 06:48 PM, Alex Mohr wrote:
>>  On 5/29/2015 7:28 AM, Stefan Seefeld wrote:
>>>  Python's copy module allows for objects to be copied. The protocol 
> for
>>>  this will look up special method __copy__. It seems to me that this
>>>  would trivially work for C++ objects providing a copy-constructor.
>>>  However, the copy-constructor isn't automatically bound to 
> __copy__.
>>>  While I can certainly add that in user-code, I wonder why this 
> isn't
>>>  done by Boost.Python itself.
>>>  Does anyone know the reasons for this ? Would it seem useful to add 
> that
>>>  feature ?
>> 
>>  The __copy__ method's intended semantics are to produce a 
> "shallow"
>>  copy while the __deepcopy__ method is meant to produce a "deep" 
> copy. 
>>  Given a C++ class with a copy ctor, it's hard to know which is more
>>  appropriate.  For instance, copying a shared_ptr<T> is like a
>>  "shallow" copy but copying a vector<pair<int, float>> 
> is like a deep
>>  copy.  On the other hand copying a vector<shared_ptr<T>> is 
> more like
>>  a shallow copy.
>> 
>>  I wouldn't mind an opt-in convenience utility that adds a __copy__ or
>>  __deepcopy__ method, but given the semantic subtlety, I think trying
>>  to do it automatically is dicey.
> 
> That's a fair point.
> 


I agree with Alex Mohr. Adding some mechanism of implementing the copy using copy constructor would be welcome. But doing it automatically wouldn't be ok. I think that explicit is better than implicit.

Trigve