[Numpy-discussion] A C++ library and the new array interface: the best approach?

[Andrew Straw]

Dear Fernando,

A couple of quick thoughts:

A) Despite my love of Pyrex, it probably isn't the tool for the job --
it doesn't play particularly well with C++, and it would require you to
write custom code for every function wrapped.

B) It sounds like you want something that will more-or-less
automatically generate the python interface from the header files. To my
mind, this suggests SWIG or boost. Boost uses some seriously fancy (and
mind-bending) C++, which can either be a good thing or a bad thing, but
it certainly boosts compile time! There are doubtless other ways. For
example, perhaps you could try using gccxml to auto-generate something
like a Python/ctypes wrapper.

C) The core of whatever you do will probably involve writing code to
view a Tensor as an object that has the __array_struct__ interface. That
way numpy/scipy/whatever can use the data with no copying. So that's a
good place to start and whatever you learn there will probably be useful
regardless of whatever tools you end up with. You'll probably also want
the converse function so that you can view objects exposing
__array_struct__ (numpy arrays) as a Tensor.

Anyhow, that's not much, but perhaps it'll help.

Cheers!
Andrew

Fernando Perez wrote:

>Hi all,
>
>I'd like to ask for a bit of guidance on the best path to follow regarding the 
>integration of an existing C++ library with numpy, given the recent 
>developments on these topics:
>
>1. the new array interface specification
>2. the ctypes discussions
>3. pyrex
>
>Our situation is the following: an existing in-house C++ library defines a 
>templated class (Tensor), in many ways similar to a numpy array, but with its 
>own extra functionality for other things.  There are many functions in this 
>code which take or return Tensors.  I now need to interface this library with 
>a Python code which is fully numpy-based, and hopefully be able to go back and 
>forth between the C++ and python sides without unnecessary copies.
>
>Since the C++ codebase is somewhat large and already exists, and will continue 
>to evolve as a standalone C++ system, something tells me I should just bite 
>the typemap bullet and SWIG the darn thing.  But I've never written anything 
>beyond the most trivial typemaps, and I'm not 100% sure on exactly how to take 
>advantage of the new array interface with SWIG.  I read all the docs under
>
>  - http://svn.scipy.org/svn/PEP
>
>  - http://www.scipy.org/BaseArray
>
>  - http://numeric.scipy.org/array_interface.html
>
>  - the pyrex/array interface wiki pages (the site is not responding right 
>now, so I can't give a URL)
>
>
>but my lack of familiarity with all the details of new type creation got me a 
>bit lost.  I'm sure the information I need is all there, but right now I don't 
>really see the forest with all the leaves in my face.  I've also read the 
>various recent threads on ctypes, as well was the one initiated by Joris:
>
>http://article.gmane.org/gmane.comp.python.numeric.general/6296
>
>
>So I'd like to know if SWIG is really the best way out in this particular case 
>(and any advice on taking advantage of the array interface via SWIG would be 
>appreciated), or if ctypes or pyrex could be used here.  I'm quite happy using 
>pyrex in other contexts, but I know it doesn't directly support C++.  However, 
>since I have access to all the code, perhaps a pure C layer could be used to 
>bridge the C++/pyrex gap.  Or given the recent praise for ctypes, perhaps that 
>can be an option?
>
>Any advice will be greatly appreciated.
>
>Best,
>
>f
>
>Using Tomcat but need to do more? Need to support web services, security?
>Get stuff done quickly with pre-integrated technology to make your job easier
>Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
>http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
>_______________________________________________
>Numpy-discussion mailing list
>Numpy-discussion at lists.sourceforge.net
>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
>  
>