From petr_aleksandrov at mail.ru  Tue Mar  5 12:13:12 2013
From: petr_aleksandrov at mail.ru (=?UTF-8?B?0JDQu9C10LrRgdCw0L3QtNGA0L7QsiDQn9C10YLRgA==?=)
Date: Tue, 05 Mar 2013 15:13:12 +0400
Subject: [C++-sig] [Boost.Python] Adding a constructor to python object
Message-ID: <5135D348.2040809@mail.ru>

I want to create a Python interface for 
"boost::numeric::ublas::vector<TReal>" where TReal is a C++ class for 
numbers. How can I add my constructor (to create a vector from a python 
list) to the python interface for "ublas::vector"? The following approach:

class_<CVector>("vector")
     .def(init<size_t>())
     .def(init<list>())
     .def("__getitem__", &vector_get_item)
     .def("__setitem__", &vector_set_item)
     .def("resize", &CVector::resize)
     .def("size", &CVector::size);

where

class CVector: public ublas::vector<TReal> {
public:
      ...
      CVector(const list &l) { ... }
};

doesn't works. The python program exits with the following message

Boost.Python.ArgumentError: Python argument types in 
some_function(some_class, vector) did not match C++ signature: 
some_function(..., boost::numeric::ublas::vector<...>)

I think that boost doesn't understand that "CVector" is practically 
"ublas::vector<TReal>". Inheriting "CVector" from 
"wrapper<ublas::vector<TReal> >" solves this problem. But I need to add 
"boost::noncopyable" to "class_<CVector>" while I want to use a function 
that returns "ublas::vector<TReal>" in python. Is it possible not to add 
"boost::noncopyable" in such case? Numpy bindings are not appropriate 
because I use "ublas::vector" with custom type for real numbers.

From ndbecker2 at gmail.com  Tue Mar  5 13:11:25 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Tue, 05 Mar 2013 07:11:25 -0500
Subject: [C++-sig] [Boost.Python] Adding a constructor to python object
References: <5135D348.2040809@mail.ru>
Message-ID: <kh4nda$4i5$1@ger.gmane.org>

??????????? ???? wrote:

> I want to create a Python interface for
> "boost::numeric::ublas::vector<TReal>"
Not a direct answer to your question, but you might like pyUblas


From petr_aleksandrov at mail.ru  Fri Mar  8 13:49:27 2013
From: petr_aleksandrov at mail.ru (=?UTF-8?B?0JDQu9C10LrRgdCw0L3QtNGA0L7QsiDQn9C10YLRgA==?=)
Date: Fri, 08 Mar 2013 16:49:27 +0400
Subject: [C++-sig] [Boost.Python] Resizing boost::numeric::array
Message-ID: <5139DE57.6080006@mail.ru>

I have the C++ method which resize boost::numeric::array:

void solve(const TReal start_t, const TReal end_t, 
boost::python::numeric::array &result) {
     ...
     result.resize(make_tuple(...));
     ...
}

and this method is called from Python:
some_object.solve(0, 10, solution)

where solution = numpy.ndarray([])

In C++ file (where Python module is defined) I use
numeric::array::set_module_and_type("numpy", "ndarray");

The program exits with the following message:

Traceback (most recent call last):
   File ...
     some_object.solve(0, 10, solution)
ValueError: cannot resize an array references or is referenced
by another array in this way.  Use the resize function

How can I resize the boost::numeric::array?

From talljimbo at gmail.com  Thu Mar 14 15:39:23 2013
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 14 Mar 2013 10:39:23 -0400
Subject: [C++-sig] [Boost.Python] Patch to add post-initialization hooks
 for value_holder
In-Reply-To: <CAOXNxjVZRsxqQJA+b5dcqN0DUsQbhpog=rXbaGUs=v0wZ=cvbg@mail.gmail.com>
References: <CAOXNxjVZRsxqQJA+b5dcqN0DUsQbhpog=rXbaGUs=v0wZ=cvbg@mail.gmail.com>
Message-ID: <5141E11B.9050506@gmail.com>

(this is a very late reply, but hopefully it will still be useful)

Your patch looks fine to me, but I'm not really sure who is responsible for approving patches and committing them to the Boost.Python codebase these days.  I suspect the procedure for doing so is a little bit in flux, as boost is in the process of a move from SVN to git.

I'd be willing to take on some of that responsibility myself, especially after the move to git is complete, with the caveat that I really won't have much time to devote to it.

First, however...

On 02/13/2013 03:22 PM, Aaron Wishnick wrote:

> I'd also be happy (and interested!) to discuss any alternate ways to accomplish what I want. To reiterate, it's really important to us that we be able to do this for all subclasses of a certain class, automatically, and this was the only apparent mechanism to me.
>

...I do have a possible alternative to propose: could you wrap all of your classes with a different "HeldType", as described here:

http://www.boost.org/doc/libs/1_53_0/libs/python/doc/v2/class.html

I'm imagining something like this:

template <typename Base>
class HookWrapper : public Base {
public:

     HookWrapper(PyObject * self, Base const & other); // this is invoked whenever an object is returned by value or copy-constructed in Python.

     HookWrapper(PyObject * self, ...);   // need to expose other constructors of Base, but with PyObject* as first argument.

};

When an object is *not* constructed in Python, it will just be a Base, not a Hookwrapper<Base>; that will be true for objects constructed in C++ and passed by pointer or reference to Python as well, which is (I think) what you want.

The obvious problem is that HookWrapper would have to be support all possible constructor signatures for the classes it would be used with (or at least those that need to be accessible from Python).  That's relatively easy in C++11 using variadic templates, but a giant pain in C++98.

Is there any chance this could work for you anyway?


Jim Bosch


From talljimbo at gmail.com  Thu Mar 14 15:42:56 2013
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 14 Mar 2013 10:42:56 -0400
Subject: [C++-sig] [Boost.Python] Resizing boost::numeric::array
In-Reply-To: <5139DE57.6080006@mail.ru>
References: <5139DE57.6080006@mail.ru>
Message-ID: <5141E1F0.8000903@gmail.com>

On 03/08/2013 07:49 AM, ??????????? ???? wrote:
> I have the C++ method which resize boost::numeric::array:
>
> void solve(const TReal start_t, const TReal end_t, boost::python::numeric::array &result) {
>      ...
>      result.resize(make_tuple(...));
>      ...
> }
>
> and this method is called from Python:
> some_object.solve(0, 10, solution)
>
> where solution = numpy.ndarray([])
>
> In C++ file (where Python module is defined) I use
> numeric::array::set_module_and_type("numpy", "ndarray");
>
> The program exits with the following message:
>
> Traceback (most recent call last):
>    File ...
>      some_object.solve(0, 10, solution)
> ValueError: cannot resize an array references or is referenced
> by another array in this way.  Use the resize function
>
> How can I resize the boost::numeric::array?

I'm afraid it's probably impossible in this situation.  NumPy arrays can share memory to create views, and because resizing requires the array to be reallocated, you're not allowed to do that if any views already exist - and that's impossible to guarantee if you don't control where your array is coming from.  In general, I would recommend against trying resize NumPy arrays in *any* context.  It just doesn't fit well with how arrays work under the hood, and IMO there should never have been a resize method.

Jim


From talljimbo at gmail.com  Thu Mar 14 15:57:29 2013
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 14 Mar 2013 10:57:29 -0400
Subject: [C++-sig] CallPolicy for operators
In-Reply-To: <5104FB1D.3060506@users.sourceforge.net>
References: <5104FB1D.3060506@users.sourceforge.net>
Message-ID: <5141E559.2040302@gmail.com>

On 01/27/2013 05:02 AM, Michael Wild wrote:
> Hi all
>
> Is there a way to apply a CallPolicy to operator definitions? In
> particular, I'm interested in the inplace operators (+=, -=, *=, /= and
> friends).
>
> To give a bit more context: The library I'm trying to wrap exposes some
> static const objects. So far, I have been able to wrap modifying
> function by having a registration facility for these static const
> objects and a custom CallPolicy that raises an AttributError when one
> tries to modify one of the registered objects.
>
> One option I see is to directly define __iadd__ etc. instead of using
> the built-in convenience operators. However, I'm not sure whether that
> has unintended side-effects. I haven't completely understood the
> operators implementation in bp, but it looks fairly involved.
>

I think directly implementing __iadd__ is probably your best bet.  I don't think there are any side-effects you need to worry about, but it would probably be a lot of boilerplate.  Do make sure you read up on how those methods are supposed to be implemented in pure-Python as regards to handling unexpected types, however, and make sure you follow the same rules in your C++ implementation.

Jim



From ndbecker2 at gmail.com  Thu Mar 14 23:12:24 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 14 Mar 2013 18:12:24 -0400
Subject: [C++-sig] checking passed arg is python type
Message-ID: <khti05$32s$1@ger.gmane.org>

I'm expecting a numpy.random.RandomState object to be passed to me.  I want to 
check it is.  The best I could think of is below.  Anything more boost::python-
ish to do this task?

static object rs_obj;

template<typename out_type=uint32_t>
struct engine {
  typedef out_type result_type;
  bp::object rs;		// RandomState object

  void verify (object const& o) {
    if (!PyObject_IsInstance(o.ptr(), rs_obj.ptr()))
      throw std::runtime_error ("engine: expected RandomState");
  }

  engine (bp::object const& _rs) : rs (_rs) { verify (_rs); }
...

BOOST_PYTHON_MODULE...

  object mod = object (handle<> ((PyImport_ImportModule("numpy.random"))));
  rs_obj = mod.attr("RandomState");


From talljimbo at gmail.com  Thu Mar 14 23:47:47 2013
From: talljimbo at gmail.com (Jim Bosch)
Date: Thu, 14 Mar 2013 18:47:47 -0400
Subject: [C++-sig] checking passed arg is python type
In-Reply-To: <khti05$32s$1@ger.gmane.org>
References: <khti05$32s$1@ger.gmane.org>
Message-ID: <51425393.8050108@gmail.com>

On 03/14/2013 06:12 PM, Neal Becker wrote:
> I'm expecting a numpy.random.RandomState object to be passed to me.  I want to
> check it is.  The best I could think of is below.  Anything more boost::python-
> ish to do this task?
>
> static object rs_obj;
>
> template<typename out_type=uint32_t>
> struct engine {
>    typedef out_type result_type;
>    bp::object rs;		// RandomState object
>
>    void verify (object const& o) {
>      if (!PyObject_IsInstance(o.ptr(), rs_obj.ptr()))
>        throw std::runtime_error ("engine: expected RandomState");
>    }
>
>    engine (bp::object const& _rs) : rs (_rs) { verify (_rs); }
> ...
>
> BOOST_PYTHON_MODULE...
>
>    object mod = object (handle<> ((PyImport_ImportModule("numpy.random"))));
>    rs_obj = mod.attr("RandomState");
>

I believe you can replace the penultimate line with:

bp::object mod = bp::import("numpy.random");

But I can't think of any ways to Boost.Python-ize the rest; there really should be an "isinstance" wrapper in Boost.Python, but there isn't.  You may also want to be a little careful with the return value of PyObject_IsInstance - I believe that it's psosible for it to raise an exception instead of simply returning 0, and in that case you'd want to use bp::throw_error_already_set() to propagate the exception to Python.


Jim



From ndbecker2 at gmail.com  Thu Mar 14 23:51:10 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Thu, 14 Mar 2013 18:51:10 -0400
Subject: [C++-sig] checking passed arg is python type
References: <khti05$32s$1@ger.gmane.org> <51425393.8050108@gmail.com>
Message-ID: <khtk8q$oco$1@ger.gmane.org>

Jim Bosch wrote:

> On 03/14/2013 06:12 PM, Neal Becker wrote:
>> I'm expecting a numpy.random.RandomState object to be passed to me.  I want
>> to
>> check it is.  The best I could think of is below.  Anything more
>> boost::python- ish to do this task?
>>
>> static object rs_obj;
>>
>> template<typename out_type=uint32_t>
>> struct engine {
>>    typedef out_type result_type;
>>    bp::object rs;		// RandomState object
>>
>>    void verify (object const& o) {
>>      if (!PyObject_IsInstance(o.ptr(), rs_obj.ptr()))
>>        throw std::runtime_error ("engine: expected RandomState");
>>    }
>>
>>    engine (bp::object const& _rs) : rs (_rs) { verify (_rs); }
>> ...
>>
>> BOOST_PYTHON_MODULE...
>>
>>    object mod = object (handle<> ((PyImport_ImportModule("numpy.random"))));
>>    rs_obj = mod.attr("RandomState");
>>
> 
> I believe you can replace the penultimate line with:
> 
> bp::object mod = bp::import("numpy.random");
> 
> But I can't think of any ways to Boost.Python-ize the rest; there really
> should be an "isinstance" wrapper in Boost.Python, but there isn't.  You may
> also want to be a little careful with the return value of PyObject_IsInstance
> - I believe that it's psosible for it to raise an exception instead of simply
> returning 0, and in that case you'd want to use bp::throw_error_already_set()
> to propagate the exception to Python.
> 
> 
> Jim

Thanks, that seems to work, and looks ever so much prettier.


From ndbecker2 at gmail.com  Fri Mar 15 12:54:30 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 15 Mar 2013 07:54:30 -0400
Subject: [C++-sig] CallPolicy for operators
References: <5104FB1D.3060506@users.sourceforge.net>
	<5141E559.2040302@gmail.com>
Message-ID: <khv25j$2v8$2@ger.gmane.org>

Jim Bosch wrote:

> On 01/27/2013 05:02 AM, Michael Wild wrote:
>> Hi all
>>
>> Is there a way to apply a CallPolicy to operator definitions? In
>> particular, I'm interested in the inplace operators (+=, -=, *=, /= and
>> friends).
>>
>> To give a bit more context: The library I'm trying to wrap exposes some
>> static const objects. So far, I have been able to wrap modifying
>> function by having a registration facility for these static const
>> objects and a custom CallPolicy that raises an AttributError when one
>> tries to modify one of the registered objects.
>>
>> One option I see is to directly define __iadd__ etc. instead of using
>> the built-in convenience operators. However, I'm not sure whether that
>> has unintended side-effects. I haven't completely understood the
>> operators implementation in bp, but it looks fairly involved.
>>
> 
> I think directly implementing __iadd__ is probably your best bet.  I don't
> think there are any side-effects you need to worry about, but it would
> probably be a lot of boilerplate.  Do make sure you read up on how those
> methods are supposed to be implemented in pure-Python as regards to handling
> unexpected types, however, and make sure you follow the same rules in your C++
> implementation.
> 
> Jim

I've always used e.g. __iadd__ rather than the convenience stuff.  No problems.


From themiwi at users.sourceforge.net  Fri Mar 15 13:03:18 2013
From: themiwi at users.sourceforge.net (Michael Wild)
Date: Fri, 15 Mar 2013 13:03:18 +0100
Subject: [C++-sig] CallPolicy for operators
In-Reply-To: <khv25j$2v8$2@ger.gmane.org>
References: <5104FB1D.3060506@users.sourceforge.net>
	<5141E559.2040302@gmail.com> <khv25j$2v8$2@ger.gmane.org>
Message-ID: <CAALQ5rnUa0HR8kvpbT3VZd406U+Ky5QLVW05054aXWYqiZ5L1Q@mail.gmail.com>

On Fri, Mar 15, 2013 at 12:54 PM, Neal Becker <ndbecker2 at gmail.com> wrote:

> Jim Bosch wrote:
>
> > On 01/27/2013 05:02 AM, Michael Wild wrote:
> >> Hi all
> >>
> >> Is there a way to apply a CallPolicy to operator definitions? In
> >> particular, I'm interested in the inplace operators (+=, -=, *=, /= and
> >> friends).
> >>
> >> To give a bit more context: The library I'm trying to wrap exposes some
> >> static const objects. So far, I have been able to wrap modifying
> >> function by having a registration facility for these static const
> >> objects and a custom CallPolicy that raises an AttributError when one
> >> tries to modify one of the registered objects.
> >>
> >> One option I see is to directly define __iadd__ etc. instead of using
> >> the built-in convenience operators. However, I'm not sure whether that
> >> has unintended side-effects. I haven't completely understood the
> >> operators implementation in bp, but it looks fairly involved.
> >>
> >
> > I think directly implementing __iadd__ is probably your best bet.  I
> don't
> > think there are any side-effects you need to worry about, but it would
> > probably be a lot of boilerplate.  Do make sure you read up on how those
> > methods are supposed to be implemented in pure-Python as regards to
> handling
> > unexpected types, however, and make sure you follow the same rules in
> your C++
> > implementation.
> >
> > Jim
>
> I've always used e.g. __iadd__ rather than the convenience stuff.  No
> problems.
>
>
Yeah, that's what I've been doing now.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130315/4d5b2a97/attachment.html>

From dbarak at gmail.com  Sun Mar 17 06:49:05 2013
From: dbarak at gmail.com (dbarak)
Date: Sat, 16 Mar 2013 22:49:05 -0700 (PDT)
Subject: [C++-sig] Exposing c++ class template to Python (known template
	instances)
Message-ID: <1363499345572-4644238.post@n4.nabble.com>

Assume I have this Interface:
class FooInterface
{
public:
    ...

};

class FooFilePolicy : public FooInterface
{
 ...
};

class FooIoPolicy : public FooInterface
{
 ...
};

And I have this template using the upper interface policy:

template< typename FooPolicy >
class BarWithPolicy
{
...
    FooPolicy* policy;
...
};

How can I expose BarWithPolicy to python that can be generically instantiate
with both  FooIoPolicy and FooFilePolicy ?


I don't want to do this :

BarWithPolicy<FooFilePolicy >& GetBarWithPolicyFile()
{
    return new BarWithPolicy<FooFilePolicy >;
}

BarWithPolicy<FooIoPolicy>& GetBarWithPolicyIo()
{
    return new BarWithPolicy<FooIoPolicy >;
}

BOOST_PYTHON_MODULE(PyBar)
{
    def("GetBarWithPolicyFile",
        &GetBarWithPolicyFile,
        return_value_policy<reference_existing_object>(),
        ""
        );
    def("GetBarWithPolicyIo",
        &GetBarWithPolicyIo,
        return_value_policy<reference_existing_object>(),
        ""
        );
}

I would like to have some thing like this in python:

>>> barPolicy = GetBarWithPolic(Io)
Or
>>> barPolicy = GetBarWithPolic(File)

Thanks
-Dotan





--
View this message in context: http://boost.2283326.n4.nabble.com/Exposing-c-class-template-to-Python-known-template-instances-tp4644238.html
Sent from the Python - c++-sig mailing list archive at Nabble.com.

From brandsmeier at gmx.de  Sun Mar 17 08:36:00 2013
From: brandsmeier at gmx.de (Holger Brandsmeier)
Date: Sun, 17 Mar 2013 08:36:00 +0100
Subject: [C++-sig] Exposing c++ class template to Python (known template
	instances)
In-Reply-To: <1363499345572-4644238.post@n4.nabble.com>
References: <1363499345572-4644238.post@n4.nabble.com>
Message-ID: <CAFP7zVtsQuGNfALgm3D2nvJHSyX1v6Mf3RPp66oKY9FSOdT82A@mail.gmail.com>

Dotan,

in
  >>> barPolicy = GetBarWithPolic(Io)
  >>> barPolicy = GetBarWithPolic(File)
what type of object would you imaginge `Io` and `File` to be?

You already exported the functions as `GetBarWithPolicIo` and
`GetBarWithPolicFile` and then you could implement in python
`GetBarWithPolic(str)` and there use something like `return
globals()["GetBarWithPolic%s" % (str,)]()` where you would now do
  barPolicy = GetBarWithPolic("Io")
  barPolicy = GetBarWithPolic("File")

-Holger

On Sun, Mar 17, 2013 at 6:49 AM, dbarak <dbarak at gmail.com> wrote:
> Assume I have this Interface:
> class FooInterface
> {
> public:
>     ...
>
> };
>
> class FooFilePolicy : public FooInterface
> {
>  ...
> };
>
> class FooIoPolicy : public FooInterface
> {
>  ...
> };
>
> And I have this template using the upper interface policy:
>
> template< typename FooPolicy >
> class BarWithPolicy
> {
> ...
>     FooPolicy* policy;
> ...
> };
>
> How can I expose BarWithPolicy to python that can be generically instantiate
> with both  FooIoPolicy and FooFilePolicy ?
>
>
> I don't want to do this :
>
> BarWithPolicy<FooFilePolicy >& GetBarWithPolicyFile()
> {
>     return new BarWithPolicy<FooFilePolicy >;
> }
>
> BarWithPolicy<FooIoPolicy>& GetBarWithPolicyIo()
> {
>     return new BarWithPolicy<FooIoPolicy >;
> }
>
> BOOST_PYTHON_MODULE(PyBar)
> {
>     def("GetBarWithPolicyFile",
>         &GetBarWithPolicyFile,
>         return_value_policy<reference_existing_object>(),
>         ""
>         );
>     def("GetBarWithPolicyIo",
>         &GetBarWithPolicyIo,
>         return_value_policy<reference_existing_object>(),
>         ""
>         );
> }
>
> I would like to have some thing like this in python:
>
>>>> barPolicy = GetBarWithPolic(Io)
> Or
>>>> barPolicy = GetBarWithPolic(File)
>
> Thanks
> -Dotan
>
>
>
>
>
> --
> View this message in context: http://boost.2283326.n4.nabble.com/Exposing-c-class-template-to-Python-known-template-instances-tp4644238.html
> Sent from the Python - c++-sig mailing list archive at Nabble.com.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig

From dbarak at gmail.com  Sun Mar 17 13:50:02 2013
From: dbarak at gmail.com (Dotan Barak)
Date: Sun, 17 Mar 2013 14:50:02 +0200
Subject: [C++-sig] Exposing c++ class template to Python (known template
	instances)
In-Reply-To: <CAFP7zVtsQuGNfALgm3D2nvJHSyX1v6Mf3RPp66oKY9FSOdT82A@mail.gmail.com>
References: <1363499345572-4644238.post@n4.nabble.com>
	<CAFP7zVtsQuGNfALgm3D2nvJHSyX1v6Mf3RPp66oKY9FSOdT82A@mail.gmail.com>
Message-ID: <CAF24SNs8bLRwpgRNWeHiWOhU4P_v+Xd9mCsJfkM=m+YzE_1YFw@mail.gmail.com>

Thanks for the response

That trick can help me out in the python level since python is has a duck
typing attribute.

do you familiar with any a procedure for the C++ ?. what I mean is this : I
would like to have some generic pointer that can either point to
BarWithPolicy<FooFilePolicy
> nor to BarWithPolicy<FooIoPolicy > ?


thanks
-Dotan

On Sun, Mar 17, 2013 at 9:36 AM, Holger Brandsmeier <brandsmeier at gmx.de>wrote:

> Dotan,
>
> in
>   >>> barPolicy = GetBarWithPolic(Io)
>   >>> barPolicy = GetBarWithPolic(File)
> what type of object would you imaginge `Io` and `File` to be?
>
> You already exported the functions as `GetBarWithPolicIo` and
> `GetBarWithPolicFile` and then you could implement in python
> `GetBarWithPolic(str)` and there use something like `return
> globals()["GetBarWithPolic%s" % (str,)]()` where you would now do
>   barPolicy = GetBarWithPolic("Io")
>   barPolicy = GetBarWithPolic("File")
>
> -Holger
>
> On Sun, Mar 17, 2013 at 6:49 AM, dbarak <dbarak at gmail.com> wrote:
> > Assume I have this Interface:
> > class FooInterface
> > {
> > public:
> >     ...
> >
> > };
> >
> > class FooFilePolicy : public FooInterface
> > {
> >  ...
> > };
> >
> > class FooIoPolicy : public FooInterface
> > {
> >  ...
> > };
> >
> > And I have this template using the upper interface policy:
> >
> > template< typename FooPolicy >
> > class BarWithPolicy
> > {
> > ...
> >     FooPolicy* policy;
> > ...
> > };
> >
> > How can I expose BarWithPolicy to python that can be generically
> instantiate
> > with both  FooIoPolicy and FooFilePolicy ?
> >
> >
> > I don't want to do this :
> >
> > BarWithPolicy<FooFilePolicy >& GetBarWithPolicyFile()
> > {
> >     return new BarWithPolicy<FooFilePolicy >;
> > }
> >
> > BarWithPolicy<FooIoPolicy>& GetBarWithPolicyIo()
> > {
> >     return new BarWithPolicy<FooIoPolicy >;
> > }
> >
> > BOOST_PYTHON_MODULE(PyBar)
> > {
> >     def("GetBarWithPolicyFile",
> >         &GetBarWithPolicyFile,
> >         return_value_policy<reference_existing_object>(),
> >         ""
> >         );
> >     def("GetBarWithPolicyIo",
> >         &GetBarWithPolicyIo,
> >         return_value_policy<reference_existing_object>(),
> >         ""
> >         );
> > }
> >
> > I would like to have some thing like this in python:
> >
> >>>> barPolicy = GetBarWithPolic(Io)
> > Or
> >>>> barPolicy = GetBarWithPolic(File)
> >
> > Thanks
> > -Dotan
> >
> >
> >
> >
> >
> > --
> > View this message in context:
> http://boost.2283326.n4.nabble.com/Exposing-c-class-template-to-Python-known-template-instances-tp4644238.html
> > Sent from the Python - c++-sig mailing list archive at Nabble.com.
> > _______________________________________________
> > Cplusplus-sig mailing list
> > Cplusplus-sig at python.org
> > http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Dotan Barak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130317/3d0692b1/attachment.html>

From brandsmeier at gmx.de  Sun Mar 17 14:13:32 2013
From: brandsmeier at gmx.de (Holger Brandsmeier)
Date: Sun, 17 Mar 2013 14:13:32 +0100
Subject: [C++-sig] Exposing c++ class template to Python (known template
	instances)
In-Reply-To: <CAF24SNs8bLRwpgRNWeHiWOhU4P_v+Xd9mCsJfkM=m+YzE_1YFw@mail.gmail.com>
References: <1363499345572-4644238.post@n4.nabble.com>
	<CAFP7zVtsQuGNfALgm3D2nvJHSyX1v6Mf3RPp66oKY9FSOdT82A@mail.gmail.com>
	<CAF24SNs8bLRwpgRNWeHiWOhU4P_v+Xd9mCsJfkM=m+YzE_1YFw@mail.gmail.com>
Message-ID: <CAFP7zVv4_FrBmoYWU2MGVtiJ_izZfstEavWUYvVGHziKcsCEHw@mail.gmail.com>

Dotan,

Well, you can derive your template class BarWithPolicy from a
non-template class IBarWithPolicy and maybe that interface is enough
to work with. Otherwise I usually drag the template arguments along on
the C++ side.

What exactly do you want to do with that object, that can then either
be a `BarWithPolicy<FooFilePolicy >` or `BarWithPolicy<FooIoPolicy >`?

-Holger

On Sun, Mar 17, 2013 at 1:50 PM, Dotan Barak <dbarak at gmail.com> wrote:
> Thanks for the response
>
> That trick can help me out in the python level since python is has a duck
> typing attribute.
>
> do you familiar with any a procedure for the C++ ?. what I mean is this : I
> would like to have some generic pointer that can either point to
> BarWithPolicy<FooFilePolicy > nor to BarWithPolicy<FooIoPolicy > ?
>
>
> thanks
> -Dotan
>
> On Sun, Mar 17, 2013 at 9:36 AM, Holger Brandsmeier <brandsmeier at gmx.de>
> wrote:
>>
>> Dotan,
>>
>> in
>>   >>> barPolicy = GetBarWithPolic(Io)
>>   >>> barPolicy = GetBarWithPolic(File)
>> what type of object would you imaginge `Io` and `File` to be?
>>
>> You already exported the functions as `GetBarWithPolicIo` and
>> `GetBarWithPolicFile` and then you could implement in python
>> `GetBarWithPolic(str)` and there use something like `return
>> globals()["GetBarWithPolic%s" % (str,)]()` where you would now do
>>   barPolicy = GetBarWithPolic("Io")
>>   barPolicy = GetBarWithPolic("File")
>>
>> -Holger
>>
>> On Sun, Mar 17, 2013 at 6:49 AM, dbarak <dbarak at gmail.com> wrote:
>> > Assume I have this Interface:
>> > class FooInterface
>> > {
>> > public:
>> >     ...
>> >
>> > };
>> >
>> > class FooFilePolicy : public FooInterface
>> > {
>> >  ...
>> > };
>> >
>> > class FooIoPolicy : public FooInterface
>> > {
>> >  ...
>> > };
>> >
>> > And I have this template using the upper interface policy:
>> >
>> > template< typename FooPolicy >
>> > class BarWithPolicy
>> > {
>> > ...
>> >     FooPolicy* policy;
>> > ...
>> > };
>> >
>> > How can I expose BarWithPolicy to python that can be generically
>> > instantiate
>> > with both  FooIoPolicy and FooFilePolicy ?
>> >
>> >
>> > I don't want to do this :
>> >
>> > BarWithPolicy<FooFilePolicy >& GetBarWithPolicyFile()
>> > {
>> >     return new BarWithPolicy<FooFilePolicy >;
>> > }
>> >
>> > BarWithPolicy<FooIoPolicy>& GetBarWithPolicyIo()
>> > {
>> >     return new BarWithPolicy<FooIoPolicy >;
>> > }
>> >
>> > BOOST_PYTHON_MODULE(PyBar)
>> > {
>> >     def("GetBarWithPolicyFile",
>> >         &GetBarWithPolicyFile,
>> >         return_value_policy<reference_existing_object>(),
>> >         ""
>> >         );
>> >     def("GetBarWithPolicyIo",
>> >         &GetBarWithPolicyIo,
>> >         return_value_policy<reference_existing_object>(),
>> >         ""
>> >         );
>> > }
>> >
>> > I would like to have some thing like this in python:
>> >
>> >>>> barPolicy = GetBarWithPolic(Io)
>> > Or
>> >>>> barPolicy = GetBarWithPolic(File)
>> >
>> > Thanks
>> > -Dotan
>> >
>> >
>> >
>> >
>> >
>> > --
>> > View this message in context:
>> > http://boost.2283326.n4.nabble.com/Exposing-c-class-template-to-Python-known-template-instances-tp4644238.html
>> > Sent from the Python - c++-sig mailing list archive at Nabble.com.
>> > _______________________________________________
>> > Cplusplus-sig mailing list
>> > Cplusplus-sig at python.org
>> > http://mail.python.org/mailman/listinfo/cplusplus-sig
>> _______________________________________________
>> Cplusplus-sig mailing list
>> Cplusplus-sig at python.org
>> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
>
>
>
> --
> Dotan Barak
>
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig

From dbarak at gmail.com  Sun Mar 17 20:02:27 2013
From: dbarak at gmail.com (Dotan Barak)
Date: Sun, 17 Mar 2013 21:02:27 +0200
Subject: [C++-sig] Exposing c++ class template to Python (known template
	instances)
In-Reply-To: <CAFP7zVv4_FrBmoYWU2MGVtiJ_izZfstEavWUYvVGHziKcsCEHw@mail.gmail.com>
References: <1363499345572-4644238.post@n4.nabble.com>
	<CAFP7zVtsQuGNfALgm3D2nvJHSyX1v6Mf3RPp66oKY9FSOdT82A@mail.gmail.com>
	<CAF24SNs8bLRwpgRNWeHiWOhU4P_v+Xd9mCsJfkM=m+YzE_1YFw@mail.gmail.com>
	<CAFP7zVv4_FrBmoYWU2MGVtiJ_izZfstEavWUYvVGHziKcsCEHw@mail.gmail.com>
Message-ID: <CAF24SNunfjw9u=05Nh2akiGW4JmEcRw5PEfpvZA==6jL_JpXvQ@mail.gmail.com>

It is basically a logger policy and the idea is to create cross platform
logger that can be accessed from C++ and from python.
Except from the creation point I don't rely care what is the policy and
that is why I can't drag the template argument.
I think I understand what you mean by using an base class interface to a
template class I will try this.

Thanks
-Dotan


On Sun, Mar 17, 2013 at 3:13 PM, Holger Brandsmeier <brandsmeier at gmx.de>wrote:

> Dotan,
>
> Well, you can derive your template class BarWithPolicy from a
> non-template class IBarWithPolicy and maybe that interface is enough
> to work with. Otherwise I usually drag the template arguments along on
> the C++ side.
>
> What exactly do you want to do with that object, that can then either
> be a `BarWithPolicy<FooFilePolicy >` or `BarWithPolicy<FooIoPolicy >`?
>
> -Holger
>
> On Sun, Mar 17, 2013 at 1:50 PM, Dotan Barak <dbarak at gmail.com> wrote:
> > Thanks for the response
> >
> > That trick can help me out in the python level since python is has a duck
> > typing attribute.
> >
> > do you familiar with any a procedure for the C++ ?. what I mean is this
> : I
> > would like to have some generic pointer that can either point to
> > BarWithPolicy<FooFilePolicy > nor to BarWithPolicy<FooIoPolicy > ?
> >
> >
> > thanks
> > -Dotan
> >
> > On Sun, Mar 17, 2013 at 9:36 AM, Holger Brandsmeier <brandsmeier at gmx.de>
> > wrote:
> >>
> >> Dotan,
> >>
> >> in
> >>   >>> barPolicy = GetBarWithPolic(Io)
> >>   >>> barPolicy = GetBarWithPolic(File)
> >> what type of object would you imaginge `Io` and `File` to be?
> >>
> >> You already exported the functions as `GetBarWithPolicIo` and
> >> `GetBarWithPolicFile` and then you could implement in python
> >> `GetBarWithPolic(str)` and there use something like `return
> >> globals()["GetBarWithPolic%s" % (str,)]()` where you would now do
> >>   barPolicy = GetBarWithPolic("Io")
> >>   barPolicy = GetBarWithPolic("File")
> >>
> >> -Holger
> >>
> >> On Sun, Mar 17, 2013 at 6:49 AM, dbarak <dbarak at gmail.com> wrote:
> >> > Assume I have this Interface:
> >> > class FooInterface
> >> > {
> >> > public:
> >> >     ...
> >> >
> >> > };
> >> >
> >> > class FooFilePolicy : public FooInterface
> >> > {
> >> >  ...
> >> > };
> >> >
> >> > class FooIoPolicy : public FooInterface
> >> > {
> >> >  ...
> >> > };
> >> >
> >> > And I have this template using the upper interface policy:
> >> >
> >> > template< typename FooPolicy >
> >> > class BarWithPolicy
> >> > {
> >> > ...
> >> >     FooPolicy* policy;
> >> > ...
> >> > };
> >> >
> >> > How can I expose BarWithPolicy to python that can be generically
> >> > instantiate
> >> > with both  FooIoPolicy and FooFilePolicy ?
> >> >
> >> >
> >> > I don't want to do this :
> >> >
> >> > BarWithPolicy<FooFilePolicy >& GetBarWithPolicyFile()
> >> > {
> >> >     return new BarWithPolicy<FooFilePolicy >;
> >> > }
> >> >
> >> > BarWithPolicy<FooIoPolicy>& GetBarWithPolicyIo()
> >> > {
> >> >     return new BarWithPolicy<FooIoPolicy >;
> >> > }
> >> >
> >> > BOOST_PYTHON_MODULE(PyBar)
> >> > {
> >> >     def("GetBarWithPolicyFile",
> >> >         &GetBarWithPolicyFile,
> >> >         return_value_policy<reference_existing_object>(),
> >> >         ""
> >> >         );
> >> >     def("GetBarWithPolicyIo",
> >> >         &GetBarWithPolicyIo,
> >> >         return_value_policy<reference_existing_object>(),
> >> >         ""
> >> >         );
> >> > }
> >> >
> >> > I would like to have some thing like this in python:
> >> >
> >> >>>> barPolicy = GetBarWithPolic(Io)
> >> > Or
> >> >>>> barPolicy = GetBarWithPolic(File)
> >> >
> >> > Thanks
> >> > -Dotan
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > View this message in context:
> >> >
> http://boost.2283326.n4.nabble.com/Exposing-c-class-template-to-Python-known-template-instances-tp4644238.html
> >> > Sent from the Python - c++-sig mailing list archive at Nabble.com.
> >> > _______________________________________________
> >> > Cplusplus-sig mailing list
> >> > Cplusplus-sig at python.org
> >> > http://mail.python.org/mailman/listinfo/cplusplus-sig
> >> _______________________________________________
> >> Cplusplus-sig mailing list
> >> Cplusplus-sig at python.org
> >> http://mail.python.org/mailman/listinfo/cplusplus-sig
> >
> >
> >
> >
> > --
> > Dotan Barak
> >
> > _______________________________________________
> > Cplusplus-sig mailing list
> > Cplusplus-sig at python.org
> > http://mail.python.org/mailman/listinfo/cplusplus-sig
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>



-- 
Dotan Barak
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130317/652f4738/attachment-0001.html>

From beamesleach at gmail.com  Wed Mar 20 15:46:59 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 14:46:59 -0000
Subject: [C++-sig] Segfaults in object deallocation
Message-ID: <op.wt810lba8s54ea@ns1.alexleach.org.uk>

Dear list,

I've started using Boost.Python to wrap a 3rd party C++ library, and  
whilst running my unit tests, the Python interpreter segfault's during  
garbage collection.

With explicit object deletion:

$ gdb -q --args python -c 'import mylib; obj = mylib.MyObj(); del(obj)'
...
*** Error in `/usr/bin/python': free(): invalid next size (fast):  
0x0000000000806fd0 ***
======= Backtrace: =========
/usr/lib/libc.so.6(+0x7ab06)[0x7ffff74d3b06]
/usr/lib/libc.so.6(+0x7b883)[0x7ffff74d4883]
/usr/lib/libboost_python.so.1.53.0(_ZN5boost6python15instance_holder10deallocateEP7_objectPv+0x15)[0x7ffff004f555]
/usr/lib/libboost_python.so.1.53.0(+0x265a1)[0x7ffff004f5a1]
...
======= Memory map: ========
...


or, leaving it to the garbage collector:-

$ gdb -q --args python -c 'import mylib; obj = mylib.obj() '
...
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7b2b0b0 in visit_decref () from /usr/lib/libpython2.7.so.1.0
(gdb)

Quick question: What do I need to do to fix this?

==========================
--- The wrapper class  ---

The wrapper class I've written is fairly simple; I haven't explicitly  
added a 'PyObject* _self' attribute, nor done any of the wiki  
recommendations[3] for managing ownership, as I don't see anywhere in the  
tutorial that recommends any of these methods.

------------------------------------
--- Using boost::python::wrapper ---

The wrapped class (in 3rd party library) does have one virtual method  
(that is protected and I haven't exposed to Python) and a virtual  
destructor. I gather the right thing to do with virtual classes is to also  
inherit from 'boost::python::wrapper<Base>', but this doesn't affect the  
seg-fault. I guess I don't need to inherit from wrapper<Base>, as I am not  
exposing any virtual methods, nor using 'this->get_override(...)' anywhere  
in the class definition.

Another very similar class is in the same source file though (virtual  
destructor and one unexposed virtual method), but this one has a public  
copy constructor. If I inherit from 'wrapper<Base>', I get a compilation  
error regarding an argument mismatch when Boost.Python passes the derived  
class to the copy constructor of the base class. Of course I can add  
noncopyable, to the class_<..> registration, but I'd like the exposed  
class to stay copyable, and I might like to use some of the wrapper  
template class' functionality. Should I wrap the copy constructor in the  
wrapper class too? Would it be possible to add this functionality to the  
wrapper<> template, when the class is not noncopyable?


------------------------------------
--- Calling thread-safe methods  ---

I don't think this is directly relevant, but the classes in question use  
the 3rd party library's own mutex implementations to ensure thread-safety.  
I've just got these working by writing a simple utility class that  
releases the GIL in the constructor and reacquires it in its destructor,  
using the C-Python API PyGILState_* functions[1].
The general tactic was described on the boost.python wiki[2], but the  
PyEval_(Save|Restore)Thread functions described caused the same assertion  
errors that I was getting earlier, with the pthreads mutex. This happened  
when the 3rd party library tried to lock the global mutex.

Although this is mentioned in the C-Python manual, I would have found it  
useful if thread-safety was specifically mentioned in the tutorial or  
reference manual somewhere... I attach the solution I wrote to this  
problem, in case someone would want to add it and document it in  
Boost.Python somewhere.

Usage:

  {
      MakeThreadsafe scope; //< GIL acquired
      // call function that acquires pthreads mutex.
  } //< GIL released when leave scope


------------------------------------
--- Garbage collection segfaul   ---

Any help with this would be really appreciated! I thought that registering  
a wrapped class with class_<...> was enough, to ensure an object's ref  
count was incremented and decremented properly. I guess that's the  
fundamentals of the problem, but am not sure the easiest / best / most  
efficient way of solving the problem.. Again, some specific documentation,  
on when and why this problem occurs and how to solve it, in the tutorial  
would be great!

Looking forward to hearing back...

Kind regards,
Alex


[1] - http://docs.python.org/2.7/c-api/init.html#non-python-created-threads
[2] -  
http://wiki.python.org/moin/boost.python/HowTo#Multithreading_Support_for_my_function
[3] -  
http://wiki.python.org/moin/boost.python/HowTo#ownership_of_C.2B-.2B-_object

-- 
Using Opera's mail client: http://www.opera.com/mail/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: make_threadsafe.hpp
Type: application/octet-stream
Size: 977 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/dcfdacf3/attachment.obj>

From beamesleach at gmail.com  Wed Mar 20 16:41:34 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 15:41:34 -0000
Subject: [C++-sig] Extracting constant pointer to char* array from list
Message-ID: <op.wt84jkjh8s54ea@ns1.alexleach.org.uk>

Hi,

I was wondering the best way to solve this problem:-

void WrapperClass::parse_args(boost::python::list argv)
{
     int argc = boost::python::len(argv);
     const char* const* pargv = boost::python::extract<const char*  
const*>(argv); /// <----
     WrappedClass::parse_args(argc, pargv);
}

This doesn't work unfortunately. (Sorry, that's not very helpful, I know;  
I can't remember the exact error right now).

I've had a look through boost/python/extract.hpp and frankly have no idea  
how I'd extend it for this use case, but I have hacked together a rather  
bloated and repetitive solution that I copy and paste each time I have to  
do the same thing. e.g.

void WrapperClass::parse_args(const boost::python::list& argv)
{
     int argc = boost::python::len(argv);

     // - Could be a one-liner call to extract<>():-

     const char** pyargv = new const char*[argc+1];
     boost::python::stl_input_iterator<std::string> begin(argv), end;
     int i=0;
     while (begin != end)
     {
         pyargv[i++] = (*begin).c_str();
         begin++;
     }
     pyargv[i] = '\0';
     const char* const* p_argv = &pyargv[0];
     /// ----------

     WrappedClass::parse_args(argc, p_argv);
     delete[] pyargv;
}


This took quite a lot of trial and error to get running correctly, but is  
there a way to perform this type of extraction already? Could I extend  
boost::python::extract to do it for me? If so, please could someone show  
me how I could?

Cheers,
Alex


-- 
Using Opera's mail client: http://www.opera.com/mail/

From talljimbo at gmail.com  Wed Mar 20 16:52:52 2013
From: talljimbo at gmail.com (Jim Bosch)
Date: Wed, 20 Mar 2013 11:52:52 -0400
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
Message-ID: <5149DB54.6090807@gmail.com>

On 03/20/2013 10:46 AM, Alex Leach wrote:
> Dear list,
>
> I've started using Boost.Python to wrap a 3rd party C++ library, and
> whilst running my unit tests, the Python interpreter segfault's during
> garbage collection.
>
> With explicit object deletion:
>
> $ gdb -q --args python -c 'import mylib; obj = mylib.MyObj(); del(obj)'
> ...
> *** Error in `/usr/bin/python': free(): invalid next size (fast):
> 0x0000000000806fd0 ***
> ======= Backtrace: =========
> /usr/lib/libc.so.6(+0x7ab06)[0x7ffff74d3b06]
> /usr/lib/libc.so.6(+0x7b883)[0x7ffff74d4883]
> /usr/lib/libboost_python.so.1.53.0(_ZN5boost6python15instance_holder10deallocateEP7_objectPv+0x15)[0x7ffff004f555]
>
> /usr/lib/libboost_python.so.1.53.0(+0x265a1)[0x7ffff004f5a1]
> ...
> ======= Memory map: ========
> ...
>
>
> or, leaving it to the garbage collector:-
>
> $ gdb -q --args python -c 'import mylib; obj = mylib.obj() '
> ...
> Program received signal SIGSEGV, Segmentation fault.
> 0x00007ffff7b2b0b0 in visit_decref () from /usr/lib/libpython2.7.so.1.0
> (gdb)
>
> Quick question: What do I need to do to fix this?
>

It looks very much like you're deleting an object twice.  I'd guess 
that's probably because something went wrong with Boost.Python's 
understanding of ownership, but that's just a guess.  It might be due to 
threading problems.

> ==========================
> --- The wrapper class  ---
>
> The wrapper class I've written is fairly simple; I haven't explicitly
> added a 'PyObject* _self' attribute, nor done any of the wiki
> recommendations[3] for managing ownership, as I don't see anywhere in
> the tutorial that recommends any of these methods.
>

If you are returning objects by pointer or [const] reference anywhere, 
you should double-check your use of call policies, to ensure you've told 
Python about ownership correctly.  That'd be the first place I'd look 
for a problem.

The tutorial is very incomplete, and should not be used to rule out more 
advanced features you may need when wrapping more advanced libraries (I 
wish I could point you at better documentation for those features, but 
it doesn't really exist beyond what it looks like you've already found).

> ------------------------------------
> --- Using boost::python::wrapper ---
>
> The wrapped class (in 3rd party library) does have one virtual method
> (that is protected and I haven't exposed to Python) and a virtual
> destructor. I gather the right thing to do with virtual classes is to
> also inherit from 'boost::python::wrapper<Base>', but this doesn't
> affect the seg-fault. I guess I don't need to inherit from
> wrapper<Base>, as I am not exposing any virtual methods, nor using
> 'this->get_override(...)' anywhere in the class definition.
>

If you do not need to override virtual methods in Python, there's no 
need at all to use wrapper.  In that case, you're better off without it, 
as this will speed up your method calls and avoid complexity that could 
help hide your problem.

> Another very similar class is in the same source file though (virtual
> destructor and one unexposed virtual method), but this one has a public
> copy constructor. If I inherit from 'wrapper<Base>', I get a compilation
> error regarding an argument mismatch when Boost.Python passes the
> derived class to the copy constructor of the base class. Of course I can
> add noncopyable, to the class_<..> registration, but I'd like the
> exposed class to stay copyable, and I might like to use some of the
> wrapper template class' functionality. Should I wrap the copy
> constructor in the wrapper class too? Would it be possible to add this
> functionality to the wrapper<> template, when the class is not noncopyable?
>

I don't think this could be added to the wrapper<> template in C++98, as 
I think it would require inheriting a constructor.  It may not even be 
possible in C++11, because it may need to add a PyObject* argument; I 
don't exactly recall how constructors for wrapper<> work.  In any case, 
it sounds like you don't need wrapper<> at all, so I'd remove it, and 
that should make this a non-issue.

>
> ------------------------------------
> --- Calling thread-safe methods  ---
>
> I don't think this is directly relevant, but the classes in question use
> the 3rd party library's own mutex implementations to ensure
> thread-safety. I've just got these working by writing a simple utility
> class that releases the GIL in the constructor and reacquires it in its
> destructor, using the C-Python API PyGILState_* functions[1].
> The general tactic was described on the boost.python wiki[2], but the
> PyEval_(Save|Restore)Thread functions described caused the same
> assertion errors that I was getting earlier, with the pthreads mutex.
> This happened when the 3rd party library tried to lock the global mutex.
>
> Although this is mentioned in the C-Python manual, I would have found it
> useful if thread-safety was specifically mentioned in the tutorial or
> reference manual somewhere... I attach the solution I wrote to this
> problem, in case someone would want to add it and document it in
> Boost.Python somewhere.
>

http://www.boost.org/doc/libs/1_53_0/libs/python/doc/v2/faq.html#threadsupport

There are other people on this list who know a lot more about this than 
I do, but my understanding has always been that it you use Boost.Python 
with threading on the C++ side, all bets are off, unless the Python 
interaction is strictly limited to one thread.


Jim




From jaedyn.cppsig at jaedyn.co  Wed Mar 20 16:54:45 2013
From: jaedyn.cppsig at jaedyn.co (Jaedyn K. Draper)
Date: Wed, 20 Mar 2013 10:54:45 -0500
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
Message-ID: <5149DA90.6030806@jaedyn.co>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/9b1cf408/attachment-0001.html>

From beamesleach at gmail.com  Wed Mar 20 17:23:14 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 16:23:14 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <5149DA90.6030806@jaedyn.co>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DA90.6030806@jaedyn.co>
Message-ID: <op.wt86g0wm8s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 15:54:45 -0000, Jaedyn K. Draper  
<jaedyn.cppsig at jaedyn.co> wrote:

> Can you send the C++ code of MyObj? Kind of hard to diagnose a C++-side  
> crash without any C++ code.
>

Sure. Sorry, thought previous email was getting long and it might have  
been an obvious fix for someone... Just writing a smaller example for here  
now, so apologies if it doesn't compile or repreoduce the problem  
exactly... General info about the classes in this file: one's for parsing  
command line arguments (with argc and argv); the other for parsing  
environment variables from the "classical C environment pointer" envp.

Both are seg-faulting, but for brevity, just writing out the Environment  
class..

--------------------------------
-- 3rd party library classes ---

// in libraryenv.hpp

namespace library {
     class Environment
     {
         public:

             Environment(void);
             /// Constructor with the envp parameter.
             Environment(const char* const* envp);

             /// Destructor.
             virtual ~CNcbiEnvironment(void);

             /// Reset environment.
             ///
             /// Delete all cached entries, load new ones from "envp" (if  
not NULL).
             void Reset(const char* const* envp = 0);

             /// Get environment value by name.
             ///
             /// If environmnent value is not cached then call "Load(name)"  
to load
             /// the environmnent value. The loaded name/value pair will  
then be
             /// cached, too, after the call to "Get()".
             const string& Get(const string& name) const;


             /// Set an environment variable by name
             ///
             /// This will throw an exception if setting the variable fails
             void Set(const string& name, const string& value);

             /// Delete an environment variable by name
             void Unset(const string& name);

         protected:
             /// Load value of specified environment variable.
             virtual string Load(const string& name) const;

         private:
             // ....
     }
}

// in libraryenv.cpp

extern char **environ;

Environment::Environment(void)
{
     Reset(environ);
}

Environment::Environment(const char* const* envp)
{
     Reset(envp);
}

/// .....


-----------------------
-- My wrapper class ---


// in pyenv.hpp

class PyEnv
   : public library::Environment
{
public:
     PyEnv(void);
     /// Construct from a list or a dict.
     PyEnv(const boost::python::list& envp);
     PyEnv(const boost::python::dict& envp);
     virtual ~PyEnv(void);

     void ResetPyList(const boost::python::list& envp);
     void ResetPyDict(const boost::python::dict& envp);

     std::list<std::string> Filter(const std::string& prefix);
     std::list<std::string> Enumerate(void);
     boost::python::object Iterate(void);

     /* Need to wrap member functions locking pthreads mutex */
     const std::string& Get(const std::string& name) const;
     void Set(const std::string& name, const std::string& value);
     void Unset(const std::string& name);
};


// in pyenv.cpp

PyEnv::PyEnv(void)
     : Environment()
{
}

PyEnv::PyEnv(const boost::python::list& envp)
{
     MakeThreadSafe scope;
     ResetPyList(envp)
}

PyEnv::PyEnv(const boost::python::dict& envp)
{
    MakeThreadSafe scope;
    ResetPyDict(dict)
}

// other member functions..

void RegisterEnv(void)
{
     boost::python::class_<PyEnv, boost::noncopyable>
         ("Environment", boost::python::init<>() )
         .def("boost::python::init<const boost::python::list&>() )
         .def("boost::python::init<const boost::python::dict&>() )
         .def("reset", &PyEnv::ResetList )
         .def("reset", &PyEnv::ResetDict )
         .def("__getitem__", &PyEnv::Get )
         .def("__setitem__", &PyEnv::Set )
         .def("__delitem__", &PyEnv::Unset )
         .def("__iter__", &PyEnv::Iterate )
         .def("enumerate", &PyEnv::Enumerate )
         .def("filter", &PyEnv::Filter )
     ;
}




---------------------------------------

Hope that helps. I'm guessing that the problem might be fixed if I inherit  
 from boost::shared_ptr, but I have no idea why I would need to do that  
here and not elsewhere. Or perhaps I've done something wrong. I'm sure the  
iterator and enumerator methods will be broken - haven't got around to  
testing them yet - but I don't see why destroying the object would cause  
segfaults on these classes.. Any clues??

Cheers,
Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/1a28ea6d/attachment.html>

From talljimbo at gmail.com  Wed Mar 20 17:34:00 2013
From: talljimbo at gmail.com (Jim Bosch)
Date: Wed, 20 Mar 2013 12:34:00 -0400
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <op.wt86g0wm8s54ea@ns1.alexleach.org.uk>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DA90.6030806@jaedyn.co>
	<op.wt86g0wm8s54ea@ns1.alexleach.org.uk>
Message-ID: <5149E4F8.8010903@gmail.com>

On 03/20/2013 12:23 PM, Alex Leach wrote:
>
> Hope that helps. I'm guessing that the problem might be fixed if I
> inherit from boost::shared_ptr, but I have no idea why I would need to
> do that here and not elsewhere. Or perhaps I've done something wrong.
> I'm sure the iterator and enumerator methods will be broken - haven't
> got around to testing them yet - but I don't see why destroying the
> object would cause segfaults on these classes.. Any clues??
>

I don't think shared_ptr would help here (and you almost certainly don't 
want to inherit from it), though I'm curious what you're using to wrap 
std::list, as you clearly have methods that return std::list, and the 
standard library can be tricky to translate to Python due to things like 
iterator invalidation.  That said, containers of strings are much less 
tricky than containers of wrapped objects.  And given that your Python 
example that segfaults doesn't involve those methods at all, they 
probably aren't at fault.

To me, everything thing here looks pretty straightforward, except the 
multithreading...which leads me to suspect that the multithreading is 
the problem.  And that's something I'm afraid I don't know enough to 
help you with.

Jim


From beamesleach at gmail.com  Wed Mar 20 17:53:55 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 16:53:55 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <5149E4F8.8010903@gmail.com>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DA90.6030806@jaedyn.co> <op.wt86g0wm8s54ea@ns1.alexleach.org.uk>
	<5149E4F8.8010903@gmail.com>
Message-ID: <op.wt87v5hd8s54ea@ns1.alexleach.org.uk>

Thanks for your replies!

On Wed, 20 Mar 2013 16:34:00 -0000, Jim Bosch <talljimbo at gmail.com> wrote:

>
> I don't think shared_ptr would help here (and you almost certainly don't  
> want to inherit from it), though I'm curious what you're using to wrap  
> std::list, as you clearly have methods that return std::list, and the  
> standard library can be tricky to translate to Python due to things like  
> iterator invalidation.

Heh, not yet anyway! Although I've laid out the foundations, I haven't yet  
implemented my attempt at a solution, nor tested it, as am still trying to  
get rid of these seg-faults here.. I did however write a template class  
for std::list, derived from indexing_suite. It was inspired by a  
question[1] on StackOverflow, but edited to fulfill the obligations of a  
class derived from indexing_suite (in  
boost/python/suite/indexing/indexing_suite.hpp).

I've put it in a "boost_helpers" folder in my own project, but I'd be  
perfectly happy if someone wanted to put it in mainline boost. I attach it  
here, if you're interested in having it ;)


>  That said, containers of strings are much less tricky than containers  
> of wrapped objects.  And given that your Python example that segfaults  
> doesn't involve those methods at all, they probably aren't at fault.

That's what I'm thinking.. I'm not calling a method or a factory function,  
just an object's constructor and then its destructor. The base class's  
destructor is a no-op, as is the one in my derived class. I've used that  
MakeThreadSafe object in every wrapped function where the base class'  
method locks the global mutex (source attached to first email in thread).  
But I was getting these seg-faults, before I even started playing with the  
GIL, as when compiled in release mode, the pthreads assertions don't raise  
an error and execution continued to this seg-fault.

>
> To me, everything thing here looks pretty straightforward, except the  
> multithreading...which leads me to suspect that the multithreading is  
> the problem.  And that's something I'm afraid I don't know enough to  
> help you with.
>
> Jim
>


-- 
Using Opera's mail client: http://www.opera.com/mail/

From beamesleach at gmail.com  Wed Mar 20 18:06:30 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 17:06:30 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <op.wt87v5hd8s54ea@ns1.alexleach.org.uk>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DA90.6030806@jaedyn.co> <op.wt86g0wm8s54ea@ns1.alexleach.org.uk>
	<5149E4F8.8010903@gmail.com> <op.wt87v5hd8s54ea@ns1.alexleach.org.uk>
Message-ID: <op.wt88g4bn8s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 16:53:55 -0000, Alex Leach <beamesleach at gmail.com>  
wrote:

> I've put it in a "boost_helpers" folder in my own project, but I'd be  
> perfectly happy if someone wanted to put it in mainline boost. I attach  
> it here, if you're interested in having it ;)

Sorry, forgot to attach... Btw, the sort method isn't sorted yet, as I  
wanted to use the C++ STL sort algorithm, but didn't figure out how I  
could amend it to use Python's 'key' callback argument...

Cheers,
Alex
-------------- next part --------------
A non-text attachment was scrubbed...
Name: make_list.hpp
Type: application/octet-stream
Size: 10792 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/c51b0d7a/attachment.obj>

From beamesleach at gmail.com  Wed Mar 20 19:43:04 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 18:43:04 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <5149DB54.6090807@gmail.com>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DB54.6090807@gmail.com>
Message-ID: <op.wt9cx2om8s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 15:52:52 -0000, Jim Bosch <talljimbo at gmail.com> wrote:
>
> http://www.boost.org/doc/libs/1_53_0/libs/python/doc/v2/faq.html#threadsupport
>

Thanks for that link. I checked out the tnfox library, which the FAQ  
refers to as having a thread-safe implementation of invoke.hpp. tnfox,  
whatever it is, also has an extensive indexing_suite, complete with a  
template for std::list containers ;)

In case you're interested, it's in the BoostPatches.zip archive, at:-
http://tnfox.git.sourceforge.net/git/gitweb.cgi?p=tnfox/tnfox;a=tree;f=Python;hb=HEAD

Cheers,
Alex

From vanv0059 at umn.edu  Wed Mar 20 20:28:35 2013
From: vanv0059 at umn.edu (Jeffrey Van Voorst)
Date: Wed, 20 Mar 2013 14:28:35 -0500
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <mailman.18476.1363798569.2938.cplusplus-sig@python.org>
References: <mailman.18476.1363798569.2938.cplusplus-sig@python.org>
Message-ID: <514A0DE3.2040406@umn.edu>

This could be misinformation, but some of the boost libraries have 
compile time flags that depend on whether or not threading is enabled.  
As an example, I was running into issues with an invalid free() with 
respect to shared_ptr (in the cctbx project). Unfortunately, (for your 
case) the fix was to download a more recent version of the cctbx project 
that no longer exhibited this problem.

In short, the boost smart pointers, boost spirit, etc. implementations 
vary depending on the compile time options.  I don't know if 
boost.python has options.

Also, I found that with these issues it is generally easier to pinpoint 
the problem when the code is compile without optimization.

--Jeff

From beamesleach at gmail.com  Wed Mar 20 20:49:11 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 19:49:11 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <514A0DA6.8030305@gmail.com>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DA90.6030806@jaedyn.co> <op.wt86g0wm8s54ea@ns1.alexleach.org.uk>
	<5149E4F8.8010903@gmail.com> <op.wt9bpqrf8s54ea@ns1.alexleach.org.uk>
	<514A0DA6.8030305@gmail.com>
Message-ID: <op.wt9fz9p88s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 19:27:34 -0000, Jim Bosch <talljimbo at gmail.com> wrote:

> (btw, this reply seemed to only go to me rather than the list)

oh, sorry... Need to get into a habit of double-checking the reply  
address..

>
> On 03/20/2013 02:16 PM, Alex Leach wrote:
>> void PyEnv::ResetPyList(const boost::python::list& envp)
>> {
>>      int envc = boost::python::len(envp);
>>      const char** pyenvp = new const char*[envc+1];
>>      boost::python::stl_input_iterator<std::string> begin(envp), end;
>>      int i=0;
>>      while (begin != end)
>>      {
>>          pyenvp[i++] = (*begin).c_str();
>>          begin++;
>>      }
>>      pyenvp[i] = '\0';
>>      const char* const* _envp = &pyenvp[0];
>>      Environment::Reset(_envp);
>>      delete[] pyenvp;
>> }
>
> This actually looks like it could be your problem, and I think the dict  
> version has the same issue.
>
> When the stl_input_iterator makes a std::string from the Python object  
> you've passed in, it's a temporary, and when you extract a c_str() from  
> it, the pointer that's returned is invalidated as soon as the  
> std::string is destroyed (which is probably as soon as the iterator is  
> incremented).

This would probably explain an error I was getting yesterday, actually..  
Thanks! Environment::Reset uses 'strchr' to search for the "=" character,  
and prints a diagnostic message if it's not found. I kept seeing that  
message, and it showed that the char* it found was garbage. However, the  
above method completes its call to Environment::Reset without issue now  
(later got segfaults because of no NULL pointer at the end of the array).

>
> So you're passing a whole lot of invalid char* pointers into  
> Environment::Reset.  Note that they're invalid before you pass them in,  
> so it doesn't matter if Environment::Reset does a deep copy of what's  
> passed in.
>
> This wouldn't explain why you're seeing the problem in the constructor  
> instead of the destructor, but sometimes memory issues are like that.


err, isn't it the other way around? The constructor's complete  
successfully (verified with plenty of Python print and C++ printf  
statements), only the destructor seg-faults... Thanks for the help though;  
appreciated!

Kind regards,
Alex

From beamesleach at gmail.com  Wed Mar 20 20:56:34 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 20 Mar 2013 19:56:34 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <514A0DE3.2040406@umn.edu>
References: <mailman.18476.1363798569.2938.cplusplus-sig@python.org>
	<514A0DE3.2040406@umn.edu>
Message-ID: <op.wt9gckg08s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 19:28:35 -0000, Jeffrey Van Voorst <vanv0059 at umn.edu>  
wrote:

> This could be misinformation, but some of the boost libraries have  
> compile time flags that depend on whether or not threading is enabled.   
> As an example, I was running into issues with an invalid free() with  
> respect to shared_ptr (in the cctbx project). Unfortunately, (for your  
> case) the fix was to download a more recent version of the cctbx project  
> that no longer exhibited this problem.
>
> In short, the boost smart pointers, boost spirit, etc. implementations  
> vary depending on the compile time options.  I don't know if  
> boost.python has options.

Okay. Thanks for the pointer though. Most of my boost libraries were built  
with full optimisation / release settings, but I compiled a debug version  
yesterday and copied over the package manager's libboost_python.so.1.53.0  
with a debug version. Didn't bother with the other boost libs though, as I  
don't think this would break ABI compatibility between boost libs, and  
don't want to slow down my system more than necessary!

>
> Also, I found that with these issues it is generally easier to pinpoint  
> the problem when the code is compile without optimization.

Yes, true! I'm currently compiling my extension library with "-g3 -O1".  
Could add "-fno-omit-frame-pointer -fbuiltins" to that, but I think it  
would be more useful to have a debug version of Python to hand, so I'm  
currently in the process of building that. (A bit of a hassle on Arch  
Linux tbh; Ubuntu makes installing debug symbols eeaaasy!)

Cheers,
Alex

From s_sourceforge at nedprod.com  Wed Mar 20 23:07:27 2013
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Wed, 20 Mar 2013 18:07:27 -0400
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <op.wt9cx2om8s54ea@ns1.alexleach.org.uk>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>,
	<5149DB54.6090807@gmail.com>,
	<op.wt9cx2om8s54ea@ns1.alexleach.org.uk>
Message-ID: <514A331F.3971.342C9C4C@s_sourceforge.nedprod.com>

On 20 Mar 2013 at 18:43, Alex Leach wrote:

> Thanks for that link. I checked out the tnfox library, which the FAQ  
> refers to as having a thread-safe implementation of invoke.hpp. tnfox,  
> whatever it is, also has an extensive indexing_suite, complete with a  
> template for std::list containers ;)

TnFOX is a metaprogrammed platform abstraction and GUI layer for my 
next-gen operating system platform, Tn (http://www.nedprod.com/Tn/). 
It was still compiling a few years, but I'd imagine it has a bad case 
of bitrot. Either way, that's all ten year old code and design now, 
and very much an orphan isolate in that it doesn't use Boost or 
recent patterns in C++. It also, rather unhelpfully given the modern 
world, wasn't written with embedded processors in mind and assumed 
exponential growth in CPU clock rates would continue much longer than 
it has.

As it happens, I was recently employed by a major corporation on the 
basis of that next-gen platform design. As a side project I'm busy 
reimplementing parts of it in Boost and C++11 right now, so you /may/ 
see something written using Boost.ASIO and Boost.Graph hit 
Boost.Sandbox this summer with the hope of it eventually being 
accepted into Boost. I'm hoping, if it works out, that said side 
project makes possible many other useful things for C++17 and beyond.

> In case you're interested, it's in the BoostPatches.zip archive, at:-
> http://tnfox.git.sourceforge.net/git/gitweb.cgi?p=tnfox/tnfox;a=tree;f=Python;hb=HEAD

I'd imagine these are highly bitrotted by now.

You may also find a discussion in this list's archives some time ago 
about what we ought to have in Boost.Python v3 (v4?) of use. There we 
discussed the general Python<=>C++ layer implementation, and how it 
would be nice if it could be arbitrarily configurable via invoke 
policies etc.

Niall

-- 
Any opinions or advice expressed here do NOT reflect those
of my employer BlackBerry Inc.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMime.p7s
Type: application/x-pkcs7-signature
Size: 6061 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/e1964a99/attachment.bin>

From super24bitsound at hotmail.com  Thu Mar 21 00:56:53 2013
From: super24bitsound at hotmail.com (Jay Riley)
Date: Wed, 20 Mar 2013 19:56:53 -0400
Subject: [C++-sig] Problem with boost::unordered_map and map_indexing_suite
In-Reply-To: <514A331F.3971.342C9C4C@s_sourceforge.nedprod.com>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>, ,
	<5149DB54.6090807@gmail.com>, ,
	<op.wt9cx2om8s54ea@ns1.alexleach.org.uk>,
	<514A331F.3971.342C9C4C@s_sourceforge.nedprod.com>
Message-ID: <BAY174-W6204A065227A63204FEACB4EA0@phx.gbl>




I've come across two problems with some python exporting Im trying to do.
First problems is as follows. I have the following piece of python code:
            inv = ActiveGame.GetPlayerParty().GetInventory().GetItems()            print str(len(inv))            print type(inv)            inv["Thunder Core"] = 10            print str(len(inv))            for key in inv.iterkeys():                print key
The problem is that the code simply hangs at print key. That is. nothing gets printed, and my program stops responding. Everything up to that point works correctly.
inv is a boost::unordered_map<std::string, int> and it's exposed like follows:
class_<boost::unordered_map<std::string, int> >("ItemCountMapUnordered")				.def(map_indexing_suite<boost::unordered_map<std::string, int> >())				;
Inventory is wrapped like this:
class InventoryComponent : public Components::Component		{		public:                        //additional methods omitted for example			boost::unordered_map<std::string, int>& GetItems();                 private:			boost::unordered_map<std::string, int> ItemCount;                 }
class_<Items::InventoryComponent, bases<Components::Component> >("InventoryComponent", init<Entity*>())				.def("GetItems", &Items::InventoryComponent::GetItems, return_value_policy<reference_existing_object>())
When I run the python code I get the output8<class 'ItemModule.ItemCountMapUnordered'>9
and then it hangs. Nothing is printed even though the map is definitely not empty and I was able to add an item to it. Is there something obvious I'm missing? I've tried switching it to an std::map from boost::unordered_map but the result was the same.
The second problem I'm having seems directly related to trying to use a boost::unordered_map with map_indexing_suite.
I wrap a map like follows:
class_<boost::unordered_map<std::string, boost::shared_ptr<Character::BaseCharacter> > >("BasePartyMemberMap")
                .def(map_indexing_suite<boost::unordered_map<std::string, boost::shared_ptr<Character::BaseCharacter> > >())
                ;
however, when I attempt to access objects of this type I get the following error
Error   26  error C2039: 'key_comp' : is not a member of 'boost::unordered::unordered_map<K,T>' c:\programming_libraries\cpp_libraries\boost_1_49_0\boost\python\suite\indexing\map_indexing_suite.hpp  155
in this case, switching boost::unordered_map to std::map solved the issue, but I'd prefer to keep boost::unordered_map if possible.
I'd appreciate any help in fixing these issues and can provide more code if you need it. Thanks in advance

 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130320/9a56e1d1/attachment-0001.html>

From beamesleach at gmail.com  Thu Mar 21 19:26:02 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Thu, 21 Mar 2013 18:26:02 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <514A0DE3.2040406@umn.edu>
References: <mailman.18476.1363798569.2938.cplusplus-sig@python.org>
	<514A0DE3.2040406@umn.edu>
Message-ID: <op.wua6togx8s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 19:28:35 -0000, Jeffrey Van Voorst <vanv0059 at umn.edu>  
wrote:

> In short, the boost smart pointers, boost spirit, etc. implementations  
> vary depending on the compile time options.  I don't know if  
> boost.python has options.

Hi again,

So I've just rebuilt everything to be compatible with Python's  
'--with-pydebug' configure option. Was quite a bit of work, as a debug  
version of libpython is not ABI compatible with a normal build, so I also  
had to rebuild libboost_python and gdb..

I've got those all working from /usr/local now, and am getting much more  
informative backtraces, but I'm still none the wiser as to why this  
problem arises.

I also overlooked the fact that Base class constructors are always called  
in the initialiser list, whether there is one specified or not. As the  
base class' constructors go on to lock the mutex, I made a few changes to  
release and reacquire the GIL around the base class constructor. Code  
demonstrating this is below...

-----------------
--- Backtrace ---

Hopefully this might make sense to someone here.. Apologies for the  
inherent bulk of posting backtraces and code..

$ /usr/local/bin/gdb -q --args python2-dbg -c 'import gc;  
gc.set_debug(gc.DEBUG_LEAK); import mylib; env =  
Environment(["foo=bar"]);  print "done" '
... (no output from my program, only gdb stuff) ...

Program received signal SIGSEGV, Segmentation fault.
boost::python::detail::construct_pointee<boost::python::list,  
boost::python::detail::borrowed_reference_t* const>  
(storage=storage at entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2) at  
/usr/include/boost/python/detail/construct.hpp:19
19          new (storage) T(x);
(gdb) bt
#0  boost::python::detail::construct_pointee<boost::python::list,  
boost::python::detail::borrowed_reference_t* const>  
(storage=storage at entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2) at  
/usr/include/boost/python/detail/construct.hpp:19
#1  0x00007ffff4b23e1e in  
boost::python::detail::construct_referent_impl<boost::python::list const,  
boost::python::detail::borrowed_reference_t* const>  
(storage=storage at entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2)
     at /usr/include/boost/python/detail/construct.hpp:25
#2  0x00007ffff4b23e2c in  
boost::python::detail::construct_referent<boost::python::list const&,  
boost::python::detail::borrowed_reference_t*>  
(storage=storage at entry=0x7fffffffd000, x=@0x7fffffffcff8: 0x2,  
tag=tag at entry=0x0)
     at /usr/include/boost/python/detail/construct.hpp:31
#3  0x00007ffff4b23e6c in object_manager_ref_arg_from_python (x=<optimized  
out>, this=0x7fffffffd000) at  
/usr/include/boost/python/converter/obj_mgr_arg_from_python.hpp:86
#4  arg_from_python (source=<optimized out>, this=0x7fffffffd000) at  
/usr/include/boost/python/arg_from_python.hpp:70
#5  operator() (args_=<optimized out>, this=0x8d0078) at  
/usr/include/boost/preprocessor/iteration/detail/local.hpp:37
#6   
boost::python::objects::caller_py_function_impl<boost::python::detail::caller<void  
(*)(_object*, boost::python::list const&),  
boost::python::default_call_policies, boost::mpl::vector3<void, _object*,  
boost::python::list const&> > >::operator() (
     this=0x8d0070, args=<optimized out>, kw=<optimized out>) at  
/usr/include/boost/python/object/py_function.hpp:38
#7  0x00007ffff010dacb in operator() (kw=0x0, args=<optimized out>,  
this=0x8d00b0) at ./boost/python/object/py_function.hpp:143
#8  boost::python::objects::function::call (this=0x8d0090, args=0x8048d0,  
keywords=0x0) at libs/python/src/object/function.cpp:226
#9  0x00007ffff010de38 in operator() (this=<optimized out>) at  
libs/python/src/object/function.cpp:585
#10  
boost::detail::function::void_function_ref_invoker0<boost::python::objects::{anonymous}::bind_return,  
void>::invoke(boost::detail::function::function_buffer &)  
(function_obj_ptr=...) at ./boost/function/function_template.hpp:188
#11 0x00007ffff0118f23 in operator() (this=<optimized out>) at  
./boost/function/function_template.hpp:767

... lots of exception handlers ...

#97 0x00007ffff010ba65 in  
handle_exception<boost::python::objects::{anonymous}::bind_return> (f=...)  
at ./boost/python/errors.hpp:29
#98 boost::python::objects::function_call (func=<optimized out>,  
args=<optimized out>, kw=<optimized out>) at  
libs/python/src/object/function.cpp:626
#99 0x00007ffff7a36978 in PyObject_Call (func=func at entry=0x8d0090,  
arg=arg at entry=0x8048d0, kw=kw at entry=0x0) at Objects/abstract.c:2529
#100 0x00007ffff7a455ed in instancemethod_call (func=0x8d0090,  
arg=0x8048d0, kw=0x0) at Objects/classobject.c:2578
#101 0x00007ffff7a36978 in PyObject_Call (func=func at entry=0x8144e0,  
arg=arg at entry=0xa79060, kw=kw at entry=0x0) at Objects/abstract.c:2529
#102 0x00007ffff7a9bf06 in slot_tp_init (self=<optimized out>,  
args=0xa79060, kwds=0x0) at Objects/typeobject.c:5663
#103 0x00007ffff7a9bb4e in type_call (type=<optimized out>, args=0xa79060,  
kwds=0x0) at Objects/typeobject.c:737
#104 0x00007ffff7a36978 in PyObject_Call (func=func at entry=0x8cfa70,  
arg=arg at entry=0xa79060, kw=kw at entry=0x0) at Objects/abstract.c:2529
#105 0x00007ffff7aec3a8 in do_call (nk=0, na=1, pp_stack=0x7fffffffd8d8,  
func=0x8cfa70) at Python/ceval.c:4239
#106 call_function (oparg=<optimized out>, pp_stack=0x7fffffffd8d8) at  
Python/ceval.c:4044
#107 PyEval_EvalFrameEx (f=f at entry=0x7214b0, throwflag=throwflag at entry=0)  
at Python/ceval.c:2666
#108 0x00007ffff7aef539 in PyEval_EvalCodeEx (co=co at entry=0x94b880,  
globals=globals at entry=0x684770, locals=locals at entry=0x684770,  
args=args at entry=0x0, argcount=argcount at entry=0, kws=kws at entry=0x0,  
kwcount=kwcount at entry=0, defs=defs at entry=0x0,
     defcount=defcount at entry=0, closure=closure at entry=0x0) at  
Python/ceval.c:3253
#109 0x00007ffff7aef631 in PyEval_EvalCode (co=co at entry=0x94b880,  
globals=globals at entry=0x684770, locals=locals at entry=0x684770) at  
Python/ceval.c:667
#110 0x00007ffff7b0cf62 in run_mod (mod=<optimized out>,  
filename=filename at entry=0x7ffff7b38836 "<string>",  
globals=globals at entry=0x684770, locals=locals at entry=0x684770,  
flags=flags at entry=0x7fffffffdb20, arena=arena at entry=0x66f540) at  
Python/pythonrun.c:1353
#111 0x00007ffff7b0dc11 in PyRun_StringFlags (str=str at entry=0x602010  
"import gc; gc.set_debug(gc.DEBUG_UNCOLLECTABLE); import mylib; env =  
mylib.Environment([\"fo=bar\"]);  print \"done\" \n",  
start=start at entry=257, globals=0x684770,
     locals=0x684770, flags=flags at entry=0x7fffffffdb20) at  
Python/pythonrun.c:1316
#112 0x00007ffff7b0e730 in PyRun_SimpleStringFlags  
(command=command at entry=0x602010 "import gc;  
gc.set_debug(gc.DEBUG_UNCOLLECTABLE); import mylib; env =  
mylib.options.Environment([\"fo=bar\"]);  print \"done\" \n",
     flags=flags at entry=0x7fffffffdb20) at Python/pythonrun.c:969
#113 0x00007ffff7b20e45 in Py_Main (argc=3, argv=0x7fffffffdce8) at  
Modules/main.c:583
#114 0x0000000000400805 in main (argc=<optimized out>, argv=<optimized  
out>) at ./Modules/python.c:23



-----------------------
-- My wrapper class ---


// in pyenv.hpp

class PyEnv
   : public library::Environment
{
public:
     PyEnv(void);
     /// Construct from a list or a dict.
     PyEnv(const boost::python::list& envp);

     // Constructor helpers to release the GIL before calling  
base-constructor
     InitPyEnv(const boost::python::list& envp);
     // ...
private:
     MakeThreadSafe * m_gil_state;

};


// in pyenv.cpp

PyEnv::PyEnv(void)
     : Environment(InitPyEnv())
{
     delete m_gil_state;
}

PyEnv::PyEnv(const boost::python::list& envp)
     : Environment( InitPyEnv(envp) )
{
     delete m_gil_state;
}

// Constructor initialiser list helpers
const char* const* InitPyEnv(void)
{
     m_gil_state = new MakeThreadSafe();
     return environ; // declared as 'extern char**', somewhere
}

const char* const* InitPyEnv(const boost::python::list& envp)
{
     int envc = boost::python::len(envp);
     const char* const* penvp = ConvertPyListToCharArrayPointer(envc, envp);
     m_gilstate = new MakeThreadsafe();
     return penvp;
}


// other member functions ...

void RegisterEnv(void)
{
     boost::python::class_<PyEnv, boost::noncopyable>
         ("Environment", boost::python::init<>() )
         .def("boost::python::init<const boost::python::list&>() )
         // ...
     ;
}

// converters.hpp

const char* const* ConvertPyListToCharArrayPointer(
     int argc, const boost::python::list& list)
{
     const char** newarr = new const char*[argc+1];
     boost::python::stl_input_iterator<std::string> begin(list), end;
     int i=0;
     while (begin != end)
     {
         newarr[i++] = (*begin).c_str();
         begin++;
     }
     newarr[i] = '\0';
     const char* const* parr = &newarr[0];
     delete[] newarr;
     return parr;
}


----------------------
--- Interpretation ---

Having dug around in the debugger a bit, ('print *this' from branch-point  
#6), the seg-fault seems to arise when registering the following class  
initialiser:-

         .def("boost::python::init<const boost::python::list&>() )

I thought this might mean there's an issue with using a const reference to  
a derived type of boost::python::object in 'init<const bp::list&>', but  
there are still problems if I comment out those '.def' lines, and instead  
use the default constructor..


-------------------------------------
--- Default constructor backtrace ---


$ /usr/local/bin/gdb -q --args python2-dbg -c 'import gc;  
gc.set_debug(gc.DEBUG_UNCOLLECTABLE); import mylib; env =  
mylib.Environment();  print "done"'
...

done
Debug memory block at address p=0x9a89a0: API 'm'
     88 bytes originally requested
     The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected.
     The 8 pad bytes at tail=0x9a89f8 are not all FORBIDDENBYTE (0xfb):
         at tail+0: 0xff *** OUCH
         at tail+1: 0xff *** OUCH
         at tail+2: 0xff *** OUCH
         at tail+3: 0xff *** OUCH
         at tail+4: 0x00 *** OUCH
         at tail+5: 0x00 *** OUCH
         at tail+6: 0x00 *** OUCH
         at tail+7: 0x00 *** OUCH
     The block was made by call #0 to debug malloc/realloc.
     Data at p: 70 92 33 f0 ff 7f 00 00 ... 00 00 00 00 00 00 00 00
Fatal Python error: bad trailing pad byte

Program received signal SIGABRT, Aborted.
0x00007ffff74582c5 in raise () from /usr/lib/libc.so.6
(gdb) bt
#0  0x00007ffff74582c5 in raise () from /usr/lib/libc.so.6
#1  0x00007ffff7459748 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff7b0df60 in Py_FatalError (msg=msg at entry=0x7ffff7b45371 "bad  
trailing pad byte") at Python/pythonrun.c:1677
#3  0x00007ffff7a7f4de in _PyObject_DebugCheckAddressApi  
(api=api at entry=109 'm', p=p at entry=0x9a89a0) at Objects/obmalloc.c:1591
#4  0x00007ffff7a7f50d in _PyObject_DebugFreeApi (api=api at entry=109 'm',  
p=0x9a89a0) at Objects/obmalloc.c:1478
#5  0x00007ffff7a7f55c in _PyMem_DebugFree (p=<optimized out>) at  
Objects/obmalloc.c:1405
#6  0x00007ffff7a7e6ed in PyMem_Free (p=<optimized out>) at  
Objects/object.c:2336
#7  0x00007ffff010b365 in boost::python::instance_holder::deallocate  
(self_=self_ at entry=0x9a74a0, storage=<optimized out>) at  
libs/python/src/object/class.cpp:760
#8  0x00007ffff010b3b1 in boost::python::objects::instance_dealloc  
(inst=0x9a74a0) at libs/python/src/object/class.cpp:336
#9  0x00007ffff7a9a642 in subtype_dealloc (self=self at entry=0x9a74a0) at  
Objects/typeobject.c:1014
#10 0x00007ffff7a7b732 in _Py_Dealloc (op=0x9a74a0) at  
Objects/object.c:2243
#11 0x00007ffff7a7366f in insertdict (mp=mp at entry=0x684760,  
key=key at entry=0x7a6fb8, hash=-3051142320843333194,  
value=value at entry=0x7ffff7da4fa0 <_Py_NoneStruct>) at  
Objects/dictobject.c:530
#12 0x00007ffff7a76675 in PyDict_SetItem (op=op at entry=0x684760,  
key=0x7a6fb8, value=0x7ffff7da4fa0 <_Py_NoneStruct>) at  
Objects/dictobject.c:775
#13 0x00007ffff7a7a8fa in _PyModule_Clear (m=<optimized out>) at  
Objects/moduleobject.c:138
#14 0x00007ffff7b01111 in PyImport_Cleanup () at Python/import.c:445
#15 0x00007ffff7b0f2bc in Py_Finalize () at Python/pythonrun.c:454
#16 0x00007ffff7b2125c in Py_Main (argc=<optimized out>, argv=<optimized  
out>) at Modules/main.c:664
#17 0x0000000000400805 in main (argc=<optimized out>, argv=<optimized  
out>) at ./Modules/python.c:23

------------------------
--- Interpretation 2 ---

The program runs to completion and seg-faults when trying to delete the  
'env' instance from the global namespace. I haven't been able to figure  
out what attribute of the Environment class is corrupted, but something  
surely is.

I was thinking I should keep around the 'const char* const*' variable for  
the lifetime of the Environment instance, destroying it in the destructor,  
but am not really sure how to allocate such a variable, be it with one or  
more calls to 'new' or 'malloc'...

Any help here much appreciated! If there's anything else that'd be useful  
to know, please do ask!
Cheers,
Alex

From beamesleach at gmail.com  Fri Mar 22 15:33:58 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Fri, 22 Mar 2013 14:33:58 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <5149DB54.6090807@gmail.com>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DB54.6090807@gmail.com>
Message-ID: <op.wucqqwtw8s54ea@ns1.alexleach.org.uk>

On Wed, 20 Mar 2013 15:52:52 -0000, Jim Bosch <talljimbo at gmail.com> wrote:

>
> http://www.boost.org/doc/libs/1_53_0/libs/python/doc/v2/faq.html#threadsupport
>
> There are other people on this list who know a lot more about this than  
> I do, but my understanding has always been that it you use Boost.Python  
> with threading on the C++ side, all bets are off, unless the Python  
> interaction is strictly limited to one thread.
>

Hi again list,

So I'm still working on this object de-allocation seg-fault problem and  
think I've pinpointed the problem... Any advice on what to do from here  
would be really appreciated!

The class I'm trying to wrap (Environment) has the following member  
definition:-

      mutable FastMutex m_CacheMutex;

The FastMutex class is implemented in the 3rd party library, and on Linux  
at least, is essentially just a wrapper around pthread_mutex.

I wrote a stripped down version of 'Environment' yesterday, and when I  
commented out that FastMutex definition and all uses of it in  
'Environment', then the seg-fault doesn't occur. If I uncomment only the  
definition, but leave its uses commented, then the seg-fault reappears. I  
guess the above definition must also be initialising the instance, as I  
can't see it being initialised anywhere else in the class...

(Interestingly, the seg-fault doesn't occur if I compile with Intel's  
compiler instead of gcc, but that's not good enough really; valgrind will  
probably still complain).

I noticed that the 3rd party library has a 'WRITE_MUTEX_EVENT' macro,  
which was a no-op, until I recompiled it with a LOG_MUTEX_EVENTS  
definition, so, I did that...

With mutex events being logged, I can see the difference between when this  
FastMutex is defined and when its not: just before the seg-fault, the  
following call is made:-

     pthread_mutex_destroy(&m_Handle);

where 'm_Handle' is a member variable of a friend class of 'FastMutex'. On  
Linux, 'm_Handle' is essentially declared as a pthread_mutex_t.


So, in Environment's destructor, which I thought was a no-op, FastMutex's  
destructor is also being called. Now, that's a bit of a bummer. As far as  
I can tell (I'm still very new to C++; coming from Python), unlike how I  
hacked the initialiser list, I can't do any work either side of the  
chained base class destructor calls.


-------------------------------------------

In an attempt to do that though, I did try this (didn't work):-

namespace mylib {
     class MyEnv
         : public notmylib::Environment
     {
     public:
         MyEnv();
         ~MyEnv();
         // ....
     private:
         PyThreadState * _save;
     }
}

mylib::MyEnv::~MyEnv()
{
     Py_UNBLOCK_THREADS;
     Environment::~Environment();  ///< compile error
     Py_BLOCK_THREADS;
}

No doubt obvious to anyone more experienced with C++, but this raises a  
compile error...

-------------------------------------------

The next thing I thought to try and do, was to get the logic inside an  
exposed '__del__' function:-

static void mylib::DelMyEnv(MyEnv* env)
{
     PyThreadState * _save;
     _save = PyEval_SaveThread();
     delete env;                   ////<-- crash
     PyEval_RestoreThread(_save);
}

BOOST_PYTHON_MODULE(pylib)
{
     class_<mylib::MyEnv, noncopyable>("Env", init<>() )
         .def("__del__", mylib::DelMyEnv )
         ;
}

However, that also segfaults on the 'delete env' line, with gdb printing  
the message:-


Error in `/usr/local/bin/python2-dbg': free(): invalid pointer:  
0x000000000096fe50 ***


That 'invalid pointer' address is for the 'env' pointer variable. With the  
power of print statements, I can see that I get into MyEnv's destructor,  
then Environment's destructor, but before that returns, crash..

The top of the backtrace is:

#0  0x00007ffff74582c5 in raise () from /usr/lib/libc.so.6[0/9672]
#1  0x00007ffff7459748 in abort () from /usr/lib/libc.so.6
#2  0x00007ffff7497f7b in __libc_message () from /usr/lib/libc.so.6
#3  0x00007ffff749db06 in malloc_printerr () from /usr/lib/libc.so.6
#4  0x00007ffff749e883 in _int_free () from /usr/lib/libc.so.6
#5  0x00007ffff6b0e377 in mylib::DelMyEnv (env=0x96fe50) at myenv.cpp:268

Again, any help here would be really appreciated! Any way I can help you  
to help me, please let me know!!

Kind regards,
Alex

From s_sourceforge at nedprod.com  Sat Mar 23 04:02:58 2013
From: s_sourceforge at nedprod.com (Niall Douglas)
Date: Fri, 22 Mar 2013 23:02:58 -0400
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <op.wucqqwtw8s54ea@ns1.alexleach.org.uk>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>,
	<5149DB54.6090807@gmail.com>,
	<op.wucqqwtw8s54ea@ns1.alexleach.org.uk>
Message-ID: <514D1B62.30387.3F87E5AD@s_sourceforge.nedprod.com>

On 22 Mar 2013 at 14:33, Alex Leach wrote:

> So, in Environment's destructor, which I thought was a no-op, FastMutex's  
> destructor is also being called. Now, that's a bit of a bummer. As far as  
> I can tell (I'm still very new to C++; coming from Python), unlike how I  
> hacked the initialiser list, I can't do any work either side of the  
> chained base class destructor calls.

Separate C++ objects from their python wrappers. Have C++ objects 
held by shared_ptr.

Have a custom deleter on shared_ptr invoke a virtual 
int_PythonIsAboutToDelete() function before it actually calls delete. 
You see as soon as destruction begins, all virtual functions cease to 
work, so you need a predestructor stage.

Do your pre destruction cleanup like destroying mutexs in that 
virtual function. Or indeed anything which can throw exceptions, 
because destructors are (soon to be) noexcept.

Hope that helps,
Niall

-- 
Any opinions or advice expressed here do NOT reflect those
of my employer BlackBerry Inc.
Work Portfolio: http://careers.stackoverflow.com/nialldouglas/



-------------- next part --------------
A non-text attachment was scrubbed...
Name: SMime.p7s
Type: application/x-pkcs7-signature
Size: 6061 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130322/c1186558/attachment.bin>

From beamesleach at gmail.com  Sun Mar 24 20:22:02 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Sun, 24 Mar 2013 19:22:02 -0000
Subject: [C++-sig] Segfaults in object deallocation
In-Reply-To: <514D1B62.30387.3F87E5AD@s_sourceforge.nedprod.com>
References: <op.wt810lba8s54ea@ns1.alexleach.org.uk>
	<5149DB54.6090807@gmail.com> <op.wucqqwtw8s54ea@ns1.alexleach.org.uk>
	<514D1B62.30387.3F87E5AD@s_sourceforge.nedprod.com>
Message-ID: <op.wugte0e98s54ea@ns1.alexleach.org.uk>

Thanks for the reply, and apologies for the delay in mine; I've been  
struggling with classes having protected destructors. Sorted now though..

On Sat, 23 Mar 2013 03:02:58 -0000, Niall Douglas  
<s_sourceforge at nedprod.com> wrote:

>
> Separate C++ objects from their python wrappers. Have C++ objects
> held by shared_ptr.
>
> Have a custom deleter on shared_ptr invoke a virtual
> int_PythonIsAboutToDelete() function before it actually calls delete.
> You see as soon as destruction begins, all virtual functions cease to
> work, so you need a predestructor stage.

This seems to be on the right track. I had some success when calling a  
Python-exposed __del__ method, with std::auto_ptr<Environment> as  
argument. I had found - with lots of printf calls - that the C++  
destructor was being called twice. Each time, it called  
pthread_mutex_destroy, and being one too many times, that caused a  
seg-fault. Seemed like an ownership problem, and calling 'release' on  
std::auto_ptr<Environment>, in the exposed "__del__" method, got around  
that seg-fault, running without error.

I was in the middle of replying with a success story, but on further  
testing, it seemed that this solution only worked when the library was  
linked against libpython3 / libboost_python3. When linking against the  
version 2.7 counterparts, seg-faults crept back in... I couldn't  
understand why...

With that partial success, I thought that I should define a corresponding  
__new__ method, to allocate the memory for the object deleted by __del__.

Although that seems okay to me in principal, I'm completely failing to  
write a successful implementation...

Are there any boost python helpers for this? Again, any help greatly  
appreciated!


// wrapper class

namespace mylib {
     class IEnvironment
         : public notmylib::Environment
     {
         IEnvironment(const boost::python::list& envp);
         ~IEnvironment(void) {}

         // method implementing __new__ functionality?
         static boost::shared_ptr<IEnvironment>  Create(
             boost::python::object type,
             const boost::python::list *envp );
     }

     // custom deleter
     void DelEnvironment(boost::shared_ptr<IEnvironment> env);

     // pointer to start of environment array
     char** m_envp;

     // thread and gil state vars
     PyThreadState *       _save;
     PyGILState_STATE  gil_state;
}


// First function to be called in initialisation chain.
const char* const* mylib::IEnvironment::InitEnvironment(const  
boost::python::list& envp)
{
     int envc = boost::python::len(envp);
     pylib::m_envp = new char*[envc+1];
     stl_input_iterator<char*>   begin(envp)   end;
     int i=0;
     while ( begin != end )
     {
         pylib::m_envp[i++] = (*begin);
         begin++;
     }
     pylib::m_envp[i] = '\0';

     pylib::gil_state = PyGILState_Ensure();
     pylib::_save = PyEval_SaveThread();

     return &pylib::m_envp[0];
}

// Defines initialiser chain. The function body here runs last.
mylib::IEnvironment::IEnvironment(const boost::python::list& envp)
     : Environment(InitIEnvironment(boost::ref(envp)))
{
     PyEval_RestoreThread(pylib::_save);
     PyGILState_Release(pylib::gil_state);
}



// attempt at writing a __new__ method
boost::shared_ptr<mylib::Environment>    mylib::Environment::Create(
     boost::python::object cls, const boost::python::list& envp )
{
     PyTypeObject* cls_type = (PyTypeObject*)cls.ptr();
     PyObject* cls_inst  = PyType_GenericNew( cls_type, envp.ptr(),  
Py_None);
     if (PyType_Ready(cls_type) != 0)
     {
         throw error_already_set();
     }
     // lots of trial and error here. Always in error, so far...
     return boost::make_shared<pylib::IEnvironment>(
             boost::python::extract<pylib::IEnvironment>(
                 boost::python::object(boost::python::ptr(cls_inst))));
}



BOOST_PYTHON_MODULE(foo)
{
     boost::python::class_<mylib::Environment, boost::noncopyable,
                           boost::shared_ptr<mylib::IEnvironment>
         ("Environment", init<const boost::python::list&>() )

         .def("__new__", &mylib::IEnvironment::Create )
         .staticmethod("__new__")  //< necessary?

         .def("__del__", &pylib::DelEnvironment )
         ;
}


>>
> Do your pre destruction cleanup like destroying mutexs in that
> virtual function. Or indeed anything which can throw exceptions,
> because destructors are (soon to be) noexcept.
>
> Hope that helps,
> Niall


Yep, thanks!

Cheers,
Alex


-- 
Using Opera's mail client: http://www.opera.com/mail/

From Holger.Joukl at LBBW.de  Tue Mar 26 18:51:05 2013
From: Holger.Joukl at LBBW.de (Holger Joukl)
Date: Tue, 26 Mar 2013 18:51:05 +0100
Subject: [C++-sig]  question on boost.python exception mechanisms
Message-ID: <OFB56A394A.EC051173-ONC1257B3A.005D363D-C1257B3A.0062103A@LBBW.de>


Hi,

I'm wrapping a C++ library that's actually just a thin wrapper around a C
lib.

Through a dispatch() method of a Dispatcher class there's an invocation of
callbacks which get implemented on the Python side, by subclassing callback
classes exposed through Boost.Python.

Now, for performing the actual dispatch the C++ library calls into the
underlying C library.

This hasn't been a problem so far as we've been targetting Solaris using
Sun/Oracle studio
compilers.

However, on Linux compiled with GCC, if an exception gets raised in the
Python callback the
program segfaults. Obviously the C part of the vendor library has not been
compiled with
(GCC-) exception support for Linux.

Unfortunately this isn't under my control and the library provider seems
not to be willing
to make the library robust against exceptions in user callback code.

So I need to keep the Boost C++ exception that "signals" the Python
exception from passing
through the C parts.

Instead of introducing some custom error/exception notification scheme I
*think* I can simply
do this:

As I need  wrapper classes for the virtual method override mechanisms
anyway I can
just ignore/suppress the exception in the callback wrapper layer and
re-throw the exception
again in the dispatcher, provided that in between we do not return to
Python and no other
call into Python is made:

I.e.

The callback wrapper simply suppresses the C++ exception "signalled" for
the Python exception:

class CallbackWrap : public Callback, public bp::wrapper<Callback> {

    // pure virtual so no need for default implementation
    // virtual void onMsg(MsgListener* listener, Msg& msg) = 0;
    virtual void onMsg(MsgListener* listener, Msg& msg) {
        try {
            this->get_override("onMsg")(bp::ptr(listener), boost::ref
(msg));
        } catch (const bp::error_already_set& e) {
            if (PyErr_Occurred()) {
                // Do nothing: The Dispatcher  that invoked the callback
                // MUST check if a Python error has been set and re-throw
            }
        }
    }
};


The Dispatcher is then responsible for re-throwing, to not let the Python
exception
go undetected when returning to the Python world:

    // virtual Status dispatch      ();
    virtual Status dispatch() {
        Status ret;
        if (bp::override f = this->get_override("dispatch")) {
                ret = f(); // *note*
        } else {
            ret = Queue::dispatch();
        }
        if (PyErr_Occurred()) {
            // Re-throw the error that stems from the Python exception in
the dispatched callback
            bp::throw_error_already_set();
        }
        return ret;
    }


Does this sound sane?

Any hints appreciated,
Holger



Landesbank Baden-Wuerttemberg
Anstalt des oeffentlichen Rechts
Hauptsitze: Stuttgart, Karlsruhe, Mannheim, Mainz
HRA 12704
Amtsgericht Stuttgart


From beamesleach at gmail.com  Wed Mar 27 12:36:47 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 27 Mar 2013 11:36:47 -0000
Subject: [C++-sig] const_iterator type
Message-ID: <op.wulrvlnj8s54ea@ns1.alexleach.org.uk>

Hi,

I'm trying to wrap an iterable class, which returns a custom iterator  
typedef'd as const_iterator, from its begin and end methods.

When using boost::python::(iterator|iterators|range) to create an iterator  
in the exposed __iter__ function in the exposed class_ def, I get a load  
of compile errors:-


/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:166:53:  
error: no type named 'iterator_category' in 'class mylib::MyConstIterator'
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:167:53:  
error: no type named 'value_type' in 'class mylib::MyConstIterator'
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:168:53:  
error: no type named 'difference_type' in 'class mylib::MyConstIterator'
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:169:53:  
error: no type named 'pointer' in 'class mylib::MyConstIterator'
/usr/lib/gcc/x86_64-unknown-linux-gnu/4.7.2/../../../../include/c++/4.7.2/bits/stl_iterator_base_types.h:170:53:  
error: no type named 'reference' in 'class mylib::MyConstIterator'

Here's a simple example demonstrating the problem:-


// const_iterator.cpp
#include <boost/python/class.hpp>
#include <boost/python/copy_const_reference.hpp>
#include <boost/python/iterator.hpp>
#include <boost/python/module.hpp>


// External lib declaration
//

namespace notmylib {

     // forward declaration
     class MyConstIterator;

     class MyData
     {
         public:
             MyData(void);
             virtual ~MyData(void);
             // ...

             typedef MyConstIterator const_iterator;
             const_iterator begin(void) const;
             const_iterator end(void) const;
     };

     class MyConstIterator
     {
         public:
             typedef std::pair<int, int> TRange;

             // constructors
             MyConstIterator(void);
             MyConstIterator(const MyData);

             // dtor
             ~MyConstIterator(void) {}

             // copy constructor
             MyConstIterator(const MyConstIterator& iter);

             // assignment operator
             MyConstIterator& operator= (const MyConstIterator& iter);

             // Comparison operators
             bool operator==(const MyConstIterator& iter) const;
             bool operator!=(const MyConstIterator& iter) const;

             // Get Data Range
            TRange GetRange(void) const;

             // Go backwards one step
             void Rewind(void);
         private:
             MyData m_data;
             size_t m_index;
     };
}

BOOST_PYTHON_MODULE(const_iterator)
{
     boost::python::class_<notmylib::MyData,
                           boost::noncopyable>("Data")
         .def("__iter__",
             boost::python::iterator<
                 const notmylib::MyData,
                 boost::python::return_value_policy<
                     boost::python::copy_const_reference> >() )
         // ...
         ;
}


Although this doesn't have the member function definitions, so won't work  
as a complete example from Python, it completely replicates the compile  
error I get with the actual class I'm trying to wrap. Command to compile,  
using GCC 4.7.2:-

g++ -c -L/usr/lib -g -O1  -fPIC -I/usr/include/python2.7  
const_iterator.cpp -o const_iterator.o -lboost_python

I couldn't find any examples using const_iterator in the test directory,  
or elsewhere, so can't find anything to go by. I thought to manually add  
those typedef's the compilers complaining about, to a Python wrapper class  
derived from MyIterator. Is there another class I can inherit from, in  
boost or the STL, to do this for me?

Any alternative suggestions or recommendations would be appreciated!

Kind regards,
Alex

--

From beamesleach at gmail.com  Wed Mar 27 14:11:05 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Wed, 27 Mar 2013 13:11:05 -0000
Subject: [C++-sig] const_iterator type
In-Reply-To: <op.wulrvlnj8s54ea@ns1.alexleach.org.uk>
References: <op.wulrvlnj8s54ea@ns1.alexleach.org.uk>
Message-ID: <op.wulv8rli8s54ea@ns1.alexleach.org.uk>

On Wed, 27 Mar 2013 11:36:47 -0000, Alex Leach <beamesleach at gmail.com>  
wrote:

> I'm trying to wrap an iterable class, which returns a custom iterator  
> typedef'd as const_iterator, from its begin and end methods.


In case any of you like answering these types of questions on  
StackExchange sites, I've also posted it up on StackOverflow:

    http://stackoverflow.com/questions/15659610/how-to-expose-c-classes-with-const-iterator

Kind regards,
Alex

From giuseppe.corbelli at copanitalia.com  Thu Mar 28 09:37:39 2013
From: giuseppe.corbelli at copanitalia.com (Giuseppe Corbelli)
Date: Thu, 28 Mar 2013 09:37:39 +0100
Subject: [C++-sig] question on boost.python exception mechanisms
In-Reply-To: <OFB56A394A.EC051173-ONC1257B3A.005D363D-C1257B3A.0062103A@LBBW.de>
References: <OFB56A394A.EC051173-ONC1257B3A.005D363D-C1257B3A.0062103A@LBBW.de>
Message-ID: <51540153.8030408@copanitalia.com>

On 26/03/2013 18:51, Holger Joukl wrote:
>
> Hi,
>
> I'm wrapping a C++ library that's actually just a thin wrapper around a C
> lib.
>
> Through a dispatch() method of a Dispatcher class there's an invocation of
> callbacks which get implemented on the Python side, by subclassing callback
> classes exposed through Boost.Python.
>
> Now, for performing the actual dispatch the C++ library calls into the
> underlying C library.
>
> This hasn't been a problem so far as we've been targetting Solaris using
> Sun/Oracle studio
> compilers.
>
> However, on Linux compiled with GCC, if an exception gets raised in the
> Python callback the
> program segfaults. Obviously the C part of the vendor library has not been
> compiled with
> (GCC-) exception support for Linux.
>
> Unfortunately this isn't under my control and the library provider seems
> not to be willing
> to make the library robust against exceptions in user callback code.
>
> So I need to keep the Boost C++ exception that "signals" the Python
> exception from passing
> through the C parts.

How does the library handle callbacks? An event loop in a separate thread? Do 
you have to explicitly call some blocking function to run the event loop?
If it's a C lib likely the callbacks are just function pointers. Being called 
inside a C object I'd say you assumption is correct: no C++ exceptions to be 
raised inside callbacks (the lib is gcc-compiled, it has no knowledge of 
exceptions).
If you could recompile with g++ the exception would raise from the event loop, 
unless catched while calling the callback function pointer.

Sidenote: how does the C++ exception mechanism work under the hood? What 
happens if it's called inside a C compiled function?

-- 
             Giuseppe Corbelli
WASP Software Engineer, Copan Italia S.p.A
Phone: +390303666318  Fax: +390302659932
E-mail: giuseppe.corbelli at copanitalia.com

From stefanrin at gmail.com  Thu Mar 28 19:52:24 2013
From: stefanrin at gmail.com (Stefan Ring)
Date: Thu, 28 Mar 2013 19:52:24 +0100
Subject: [C++-sig] const_iterator type
In-Reply-To: <op.wulrvlnj8s54ea@ns1.alexleach.org.uk>
References: <op.wulrvlnj8s54ea@ns1.alexleach.org.uk>
Message-ID: <CAAxjCEwMDv0epMw765utawtqgNocBic7VhRymJJMdkeDrJfxgg@mail.gmail.com>

> I couldn't find any examples using const_iterator in the test directory, or
> elsewhere, so can't find anything to go by. I thought to manually add those
> typedef's the compilers complaining about, to a Python wrapper class derived
> from MyIterator. Is there another class I can inherit from, in boost or the
> STL, to do this for me?

Wy can't you add them to the iterator itself? That's where they belong.

From beamesleach at gmail.com  Fri Mar 29 13:04:01 2013
From: beamesleach at gmail.com (Alex Leach)
Date: Fri, 29 Mar 2013 12:04:01 +0000
Subject: [C++-sig] const_iterator type
In-Reply-To: <CAAxjCEwMDv0epMw765utawtqgNocBic7VhRymJJMdkeDrJfxgg@mail.gmail.com>
References: <op.wulrvlnj8s54ea@ns1.alexleach.org.uk>
	<CAAxjCEwMDv0epMw765utawtqgNocBic7VhRymJJMdkeDrJfxgg@mail.gmail.com>
Message-ID: <CABOfY3DLNmDNor_fY0jAdmOp8kgBQObd4_NH_aEii651i4mynA@mail.gmail.com>

The iterator I'm wrapping comes from a 3rd party library, over which I have
no control. However, I have been able to add those missing types to the
derived inheritor class, by also inheriting from 'iterator_facade', as per
a suggestion on StackOverflow.
Thanks for the reply btw; I have no clue when it comes to C++
standards-compliance, so all the tips and pointers are appreciated!

Cheers,
Alex


On Thu, Mar 28, 2013 at 6:52 PM, Stefan Ring <stefanrin at gmail.com> wrote:

> > I couldn't find any examples using const_iterator in the test directory,
> or
> > elsewhere, so can't find anything to go by. I thought to manually add
> those
> > typedef's the compilers complaining about, to a Python wrapper class
> derived
> > from MyIterator. Is there another class I can inherit from, in boost or
> the
> > STL, to do this for me?
>
> Wy can't you add them to the iterator itself? That's where they belong.
> _______________________________________________
> Cplusplus-sig mailing list
> Cplusplus-sig at python.org
> http://mail.python.org/mailman/listinfo/cplusplus-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20130329/6b70656c/attachment.html>