[C++-sig] pybindgen: Why do wrapped C++ containers not look more like python containers?
J. Michael Owen
mikeowen at llnl.gov
Fri Jun 19 02:58:20 CEST 2009
Hello Gustavo and Taner,
Sorry I didn't follow up on this earlier with you guys -- I've been on
travel for a while since I started looking at pybindgen and just
returned. After Gustavo pointed out the python documentation for the
sequence method slots, I thought I would take a look at modifying
pybindgen to allow me to fill in the sequence slots for CppClass much
like the numeric operator slots are handled. I've been able to do
this, and am attaching some modified pybindgen source to this message
for your perusal. With these modifications I can pretty easily expose
std::vector so that length, get and set item (index access), and
iteration work. I'll fill in more methods as needed, but this is a
start.
I took this tack 'cause in my code I have C++ objects that naturally
obey both numeric object protocols (such as +, -, etc) as well as
sequence methods, and I need to reflect those operations into python.
I want them to look like python objects as much as possible, so it's
nice to be able to use [] rather than () for indexing for instance
(it's best not to confuse your users more than necessary). :)
The way I have this working is if the user adds any of the following
methods to a CppClass ("__len__", "__getitem__", "__setitem__"), then
the corresponding slots in the sequence object protocols are filled in
in using those methods. I think this is fairly natural, since this is
how you add sequence methods to an object you define in python natively.
With my modifications I can add a container as a cppclass like so:
mod = Module("example")
mod.add_include("<vector>")
mod.add_include('"example.hh"')
std = mod.add_cpp_namespace("std")
vecint = std.add_class("vector", template_parameters=["int"],
custom_template_class_name="vector_of_int")
vecint.add_constructor([])
vecint.add_constructor([param("int", "size")])
vecint.add_constructor([param("int", "size"), param("int", "value")])
vecint.add_method("size", "int", [], custom_name = "__len__")
vecint.add_function_as_method("indexContainer", "int",
[param("std::vector<int>", "self"),
param("int",
"index")],
template_parameters =
["std::vector<int>"],
custom_name = "__getitem__")
vecint.add_function_as_method("assignToPosition", None,
[param("std::vector<int>", "self"),
param("int",
"index"),
param("int",
"value")],
template_parameters =
["std::vector<int>"],
custom_name = "__setitem__")
vecint.add_method("push_back", None, [param("int", "value")],
custom_name="append")
where "example.hh" contains a few helper methods:
#include <stdexcept>
namespace std {
//------------------------------------------------------------------------------
// Extract a value from a container.
//------------------------------------------------------------------------------
template<typename Container>
inline
typename Container::value_type&
indexContainer(Container& container,
const size_t index) {
try {
return container.at(index);
} catch (out_of_range) {
PyErr_SetString(PyExc_IndexError, "Container index out of range");
}
}
//------------------------------------------------------------------------------
// Assign to a postion in a container.
//------------------------------------------------------------------------------
template<typename Container>
inline
void
assignToPosition(Container& container,
const size_t index,
const typename Container::value_type& value) {
if (index >= container.size()) {
PyErr_SetString(PyExc_IndexError, "Container index out of range");
} else {
container[index] = value;
}
}
}
And the resulting vector<int> can be used from python kind of like you
would naturally expect:
alastor1{owen}77: python
Python 2.6.1 (r261:67515, Jun 2 2009, 15:40:32)
[GCC 4.3.2 20081007 (Red Hat 4.3.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import example
>>> v = example.std.vector_of_int(20, 5)
>>> len(v)
20
>>> [x for x in v]
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5]
>>> v[10] = 100
>>> [x for x in v]
[5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 100, 5, 5, 5, 5, 5, 5, 5, 5, 5]
>>>
In this process I've also made a few other modifications to pybindgen:
1. In CppClass I've generalized the binary_numeric_operators to allow
the user to specify python types as one of the arguments. I needed to
do this so I could allow types to define "*" with a python float for
instance, rather than only allowing operators with other CppClass
wrapped objects.
2. I've also added inplace_numeric_operators as an option for
CppClass, which fills in the appropriate slots in the number methods
struct.
3. I added name mangling for template arguments, since some of my
template arguments for methods are themselves templates (such as in
the example above).
The changes I have are made against pybindgen-0.10.0, and you should
compare the files in this tarball to that version. I'm no Python C-
API whiz, so I won't claim these are the most optimal ways to handle
these extensions to pybindgen, but it does work and passes all of
pybindgen's internal tests. I'm continuing to look at how I have to
modify pybindgen in order to meet the needs of wrapping my code, but I
don't want to wander too far afield of the mainline release! If these
kinds of mods are not going to work for want you want to do Gustavo,
please let me know! Anyway, check out these mods as you can and let
me know what you think.
Mike.
On Jun 5, 2009, at 10:00 AM, Gustavo Carneiro wrote:
>
>
> 2009/6/5 J. Michael Owen <mikeowen at llnl.gov>
> Hi Taner,
>
> Thanks for the suggestion! I actually was trying the same solution,
> except I wasn't sure how to handle the iterators. I'll try it out
> as you do in your example -- are you able to iterate over the vector
> in python the usual way? As in "for x in vec:"?
>
> Two curious things I noted about wrapping "std::vector" myself:
>
> 1. I can't seem to get indexing access to work, even though I wrap
> an indexing operation with the python name "__getitem__"
>
> 2. I also can't get python's len function to work, even if I
> provide "__len__" as follows:
> veci.add_method("size", "int", [], custom_name="__len__")
>
> Have you been able to make either of those methods work?
>
> It won't work because those are special methods in C/C++ side;
> Python extension types use a "slots" mechanism for this kind of
> thing, see http:// www. python.org/doc/2.6/c-api/typeobj.html
>
>
> Thanks!
>
> Mike.
>
> --
> Gustavo J. A. M. Carneiro
> INESC Porto, Telecommunications and Multimedia Unit
> "The universe is always one step beyond logic." -- Frank Herbert
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090618/473db0c7/attachment-0002.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pybindgen-0.10.0-mods.tbz
Type: application/octet-stream
Size: 33517 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090618/473db0c7/attachment-0001.obj>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/cplusplus-sig/attachments/20090618/473db0c7/attachment-0003.htm>
More information about the Cplusplus-sig
mailing list