[C++-sig] interfacing headle binary file library, raw pointer and buffer

[heroxbd at gmail.com]

Dear all,

I am trying to interface a binary file library using py++.

There is a function in the library like

      int read(void* buffer, size_t len)

which is hard to wrap. It read a piece (size len) of the file, and store
it in buffer, and then return the buffer length (most likely equal to len).

If I do nothing special, the py++ generated module would be like:

,----
| >>> s = ""
| >>> l = 1
| >>> binary_file.read(s, l)
| ....
|  did not match C++ signature:
| (python str type could not match void*)
`----

Here are what I have tried (with no success):

1. I found the keyword "opaque pointer" in boost.python and py++. My
knowledge is so limited that I couldn't figure out if my case is
_actually_ an opaque pointer problem, cuz all the documentation[1] for
py++ and boost.python are talking about opaque return pointer, not a
passed pointer. I poked around py++ to add some class policies (from the
example of gmp wrapper) resulting in no change in the py++ generated
boost.python code.

2. Then I tried to change the function to

      int read(char* buffer, size_t len)

with type casting done internally in c++. Then

,----
| >>> s = ""
| >>> l = 1
| >>> binary_file.read(s, l)
| 1
| (the function executed successfully, with no c++ signature type mismatch)
| >>> s
| Segment Fault (python crash)
`----

maybe it is the methodology of python that not touching "low level" mem
management. I guess that's what making this problem non-trivial.

I may recompile my python with debugging information an trace it with
gdb to find out why it is crashing. But not yet.

3. I tried ctypes:

,----
| >>> s = ctypes.c_char_p
| >>> l = 1
| >>> binary_file.read(s, l)
| (c++ signature type mismatch)
`----

I guess we just cannot mix ctypes with boost.python directly.


Now I need to learn and try something new to solve this issue. After
some web searching, I find some possiblities:

1. py++ ctypes integration[2]. It looks promising. I would like to
confirm on this list if it is suited to my case.

2. write a wrapper like

,----
| struct void_p {
|    void * ptr
| }
`----

and then change all the original c++ interface from func(void * buf,
...) to func(void_p buf, ...) (there are around 6 such functions) and
take care of the internal convertion by hand. But in this case the
interface becomes thicker. I regard it as the last choice cuz I want to
keep new c++ code minimum (hopefully all auto generated by py++).

3. I came across struct module[3] in python. It seems to suit my
needs; I'm not sure. I couldn't figure out how to use it right now.

4. pyOgre might have been already solved this kind of raw memory
problem. I could learn pyOgre team's example and apply their method to
my case. There may be a sharp learning curve though.


... I am not sure which path I should take.

I am a beginner in c++ and python and c++/python integration. The
documentations and mailing list archives seems overwhelming to me. I
couldn't find a fresh point to start digging into.

Please give me some suggestion and advice. I am ready to learn and try
and practise.

Cheers!
Benda

footnotes:

1. http://www.boost.org/doc/libs/1_43_0/libs/python/doc/v2/opaque.html
   http://language-binding.net/pyplusplus/documentation/functions/call_policies/call_policies.html

2. http://language-binding.net/pyplusplus/documentation/ctypes/ctypes_integration.html

3. http://docs.python.org/library/struct.html
-- 
XU Benda
Research Center for Neutrino Science
Tohoku University
JAPAN

http://www.awa.tohoku.ac.jp/~benda