[C++-sig] Re: Interchanging SWIG & Boost bindings.

[David Abrahams]

Jacek Generowicz <jacek.generowicz at cern.ch> writes:

> Clearly, the first block of tests (self-consistency) just works.
>
> The second block (pure Python as receiver) also works, because the
> Python implementation of the bar.baz method is happy to receive and
> use _any_ object which responds to the "square()" message (signature
> polymorphism).
>
> The remaining blocks fail, however, because the Boost and SWIG
> implementations of bar.baz only accept objects of type "C++ foo
> wrapped by Boost" and "C++ foo wrapped by SWIG", respectively.
>
> There are two different issues at play; two different levels at which
> one can approach the problem.
>
> The first is that the Boost and SWIG wrappings do not recognize that
> they are actually wrapping the _same_ underlying C++ class. Would it
> be possible to make a Boost-wrapped foo look sufficiently like a
> SWIG-wrapped foo, in order for the SWIG-wrapped bar.baz to accept it
> as an argument, and to know how to access the underlying C++ object

You've got to ask the SWIG guys about that part.

> (and vice versa)? 

Yes, the vice versa one we can cover.  You just need to register an
appropriate lvalue from-Python converter.

> I'm thinking along the lines of some converter utility which takes
> Boost/SWIG-wrapped objects and transforms them into ones which look
> like SWIG/Boost-wrapped ones, _without_ need to re-create the
> original bindings.

That sentence is meaningless to me, sorry.  What's the difference
between a Boost/SWIG-wrapped object and a SWIG/Boost-wrapped one?
What does "re-create the original bindings" mean?

> The second, more general, approach could be to try to loosen the type
> checking in the Boost and SWIG wrapped versions, in order to make them
> respect signature polymorphism.

If you want full signature polymorphism, you should just re-code your
C++ functions to use boost::python::object arguments.

> The latter is of less direct interest to me, but I suspect that it
> might provide a simpler and more general way of achieving the
> former. In particular, if we (see below for a brief discussion of
> what "we" means) can agree, a priori, on a recommended way of
> creating Boost and SWIG bindings in a way that makes them respect
> signature polymorphism, later interchangeability comes for free.
>
> The broader context of my question is the following: It has been
> agreed[+] that Python should be used as the scripting language, glue
> language, and "softawre bus" of the physics applications used in the
> LHC[*] computing project. The number of developers involved is huge;
> numerous Python bindings already exist in both Boost and SWIG, as do
> long-established (and often unshakeable :-) preferences for one or
> other of the two packages. We expect (numerous developers) to create
> many new bindings in the near future (using both Boost and SWIG), and
> we expect the underlying implementations of these packages to overlap
> in some (many) places.
>
> The end-user Physicists are being encouraged to drive their
> applications in Python, with the selling point that they will be able
> to access, interactively, all the software that they need, via the
> bindings which are being made, and make various disjoint packages talk
> to eachother. What we are trying to anticipate (and avoid) is the
> situation where an end-user is happily playing with his data in an
> interactive session, suddenly to be confronted with the message:
>
>    Type error: Expected _p_Histogram
>
> when he is passing a histogram to a function which expects a
> histogram. If at this point he asks the developers for advice and gets
> told that "If you want to get histograms out of package FOO and use
> them in package BAR, then you have to wait for the authors of FOO to
> to make a second binding of FOO in SWIG ..." (by this time he has
> fallen asleep and doesn't hear the rest of the explanation), then that
> would be a BAD THING (TM). 

Understood.

> However, if we can offer the response "Just convert your histogram
> with the function Boost_to_SWIG before you pass it in"

Unlikely to be possible in that direction, unfortunately (though I'm
not familiar with SWIG internals).  SWIG_to_Boost is not really
possible as a Python function either, at least, not without
additional arguments.

> (or, better still, prevent the situation from arising in the first
> place)

That one seems more likely to be plausible.

> then the incident does not throw a huge spanner in the works.
>
> So, my questions are (sorry it took so long to get to the point!), is
> there any prior art in this area?

Not that I know of.

> Do you foresee any show-stopping problems?

Only the obvious one: we have two really big somewhat independent
projects already and those of us who are responsible for them may not
have the free time and/or energy to integrate them.

> Do you see any obvious ways of doing it? (I haven't looked into the
> grubby details myself in depth yet, but I didn't want to find myself
> re-inventing the wheel, so I thought I'd ask a broad question early
> on.)

No, this isn't simple.  I can't speak for what it would take to get
SWIG to accept Boost.Python-wrapped classes.  I can imagine what it
would take to get Boost.Python to accept SWIG-wrapped classes; there
are a few possible approaches but they'd all require a little
research into SWIG.

-- 
Dave Abrahams
Boost Consulting
www.boost-consulting.com