[C++-sig] weave plus boost?

[eric]

> ----- Original Message -----
> From: "eric" <eric at enthought.com>
>
> > Hey group,
> >
> > I've just signed up to listen in on the group.  I haven't followed boost
> to
> > closely, but am interested in catching up with what this community is
> doing.
> >
> > My main interest here is to see if there is anyway that weave can
benefit
> > from the boost effort -- I'm almost sure there is.  weave
> > (www.scipy.org/site_content/weave) is a tool for, among other things,
> > inlining C/C++ within Python.  It also has a module called ext_tools
that
> > allows you to build C/C++ extension modules using Python.  Currently,
> > ext_tools only supports building modules that contain functions.  I'd
like
> > to eventually extend it so that it handles extension classes as well.
It
> > looks like it won't be to bad, but boost might make some aspects of this
> > more elegant and easier for the end user.
>
> Possibly so.
>
> > Also, currently, weave converts Python containers to CXX List, Tuple,
and
> > Dict class objects.  I'm strongly considering switching to SCXX because
> they
> > are quite a bit simpler and more portable.  CXX is a little heavier duty
> > than weave really needs.  I expect boost provides another alternative
> here.
>
> Yes, it does, although there's no documentation for that part of
> Boost.Python yet. The framework for manipulating the built-in Python types
> is currently similar to the facility you get from CXX, but Arlnaldur
> Gylfason is working on a new design.
>
> A great strength of Boost.Python is its ability to provide extensible
> automatic type conversions between Python and C++ without a separate
> compilation step. When applied to objects like lists and dicts is that, if
> you've got to_python converters for, say my_class and std::vector<double>,
> you can write:
>
> int f(myclass const& x, std::vector<double> const& y)
> {
>     tuple three(1, x, y);
>     std::string msg("this is a test");
>     dict d;
>     d[msg] = 5;
> }

That looks very nice -- especially the d[msg] = 5 like syntax.  Can you
initialize a tuple, dict, list, etc. from a PyObject*?  If so, they are
probably close to interchangeable with the current CXX implementation I use,
and it would be easy to create a boost backend to weave.  How far along are
these basic objects
in development?  Can they handle assignments (like the d[msg]=5 statement)
from int,float,strings, etc. now?  Also, how is the speed compared to using
Python API calls?  I found that using the nice features of CXX (indexing,
assignment, etc) in innerloops was hard on performance.

>
> ...etc.
>
> Boost.Python has a different paradigm than weave does.

Right.  My understanding is that boost is CXXish (my main point of
reference) in flavor  -- i.e. made for writing stand alone C++ libs that are
usable from Python.  weave, on the other hand, takes a "build it entirely in
Python then optimize the hot spots approach".

weave has to use something underneath, though, to convert from Python to
C/C++.  This could be Python API calls (and many would rather have this),
but that results in not so pretty code that can intimidate people unfamiliar
with the API, ref counting, and error handling.  CXX handles most of this
under the covers.  I'm guessing boost does also.  I think the benefit is
great.  The code you have above, in fact,
would be familiar to any Python user, so it would be a small learning curve
to write a simple inline
snippet where they needed it using boost within weave.

> A principal idea of
> Boost.Python is that you should be able to easily expose a standalone
> library of C++ types and functions to Python without writing lots of
"glue"
> (e.g. argument checking and translation) and without intrusive changes to
> the library. Weave's orientation seems to be towards writing C++ bodies
for
> Python functions. It should be very quick for those who don't want to
leave
> their Python editor just to optimize a function. However, it seems like
that
> approach wouldn't be very compatible with the idea of a standalone C++
> library unless you just plan to write little wads of glue code in weave
and
> dispatch to your library.

Right.  This would be doable -- though with tools around that already handle
this problem (to various levels of satisfaction), I haven't seen a
compelling reason yet to head off in that direction (although this hasn't
always stopped me before...).

> It also seems as though it might not scale well,
> since (and I'm guessing here) you have to cross a Python function boundary
> every time you want to break off a new piece of functionality.

This is true.  There are ways around it to some extent.  You can define pure
C/C++ functions/classes that a weave.inline function, or group of ext_tools
functions all use.  These guys can call each other at the speed of C/C++
without the Python overhead.

> I'm not
> saying one or the other paradigm is best - they seem to serve different
> needs.

yes.

> Whether weave can benefit somehow from Boost.Python isn't clear to
> me, but I hope what I've said helps to give you an idea.

I'll watch with interest.  Judging from your little example, I think it
might.  Also, as weave ventures out into trying to build extension classes
in the future using 2.2's new additions, a nice framework that already
handles the hard stuff in C++ would definitely help.

see ya,
eric