[pypy-dev] PyPy, C/C++ and Wrappyr

[Vincent den Boer]

Hello Wim,
>
> several years ago, I did the something similar (more limited) based on
> gccxml, so I'm wondering what are the parts that you found ignored by that
> tool?
>

Somehow, for some classes their copy constructor was exported, but their 
'normal' constructors were not. I couldn't find the cause, but when I parsed the 
.cpp implementation files individually, the missing constructors showed up again. 
Because it wouldn't have been very practical to require the user to have access 
to the .cpp files, and because one big XML was easier for me to process, I 
decided to create the Clang plugin.

> An alternative to using clang to produce XML, is to use its PCH directly.
> This is the approach taken by cling (a C++ interpreter).

That would be nice, because that would mean less C++ code to maintain and users 
wouldn't need to go through the trouble of getting the Clang plugin running. 
However, it would require me to write code to parse those files. Maybe I could 
also use this:
https://llvm.org/svn/llvm-project/cfe/trunk/bindings/python/clang/cindex.py
which is the ctypes-based API to
http://clang.llvm.org/doxygen/Index_8h_source.html

> In cppyy, we keep the generated reflection information available, rather
> than wrapping the C++ into C. The advantage of that approach is that the
> complex C++ types can than be "unboxed" for the JIT. The disadvantage is,
> of course, that it's much more work to deliver and that is taking a long
> time, but we're getting there.

I've read about it in the PyPy blog, but since I didn't hear anything about it 
for a long time I forgot about it. It seems like we have two different goals. 
While cppyy purely focuses on C++, Wrappyr tries to make the path from C or C++ 
to a working Python API (compatible with any Python implementation with a 
working ctypes module) as short and simple as possible.

> This is like the SWIG .i files? Or am I misunderstanding? An alternative is
> to add rules to your XML generator for common mappings (eg. C++ STL
> iterator can be mapped on Python's iter protocol).

Those XML file are just helpers for working with ctypes. See:
https://github.com/ShishKabab/Wrappyr/blob/master/ctypes_builder/README.rst

While SWIG knows what C and C++ is, this XML file allows you to group arbitrary C 
calls into classes and describe how the resulting modules and packages should be 
organized. This has advantages over generating the ctypes code directly from the 
Clang data conversion script. The first is that it is easier to generate since 
you don't have to worry about the code itself. The second is that you can write 
scripts to preprocess the data in the XML programmatically, so you can do things 
like grouping functions into classes (which helps when the XML was generated 
from C code).

As for mapping C/C++ idioms to Python (like the __iter__ protocol), the problem 
is that many libraries have their own way of doing things. Many libraries for 
example have their own string classes/functions, iterators, hash tables, etc. It 
would be nice have a way to indicate those relationships. However, I consider 
this a lower priority for now.

Kind regards,
Vincent